Deep learning-based localisation of combine harvester components in thermal images

<jats:p>It is crucial to identify defective machine components in production to ensure quality. Some components generate heat when defective, so automating the inspection process with a thermal imaging camera can provide qualitative measurements. This work aims to use computer vision methods to locate these components in thermal images. Since there is currently no comparison of object detection and semantic segmentation algorithms for this use case, this study compares different architectures with the goal of localising these components for further defect inspection. Moreover, as there are currently no datasets for this use case, this study contributes a novel annotated dataset of thermal images of combine harvester components. The different algorithms are evaluated based on the quality of their predictions and their suitability for further defect inspection. As semantic segmentation and object detection cannot be directly compared with each other, custom weighted metrics are used. The architectures evaluated include RetinaNet, YOLOV8 Detector, DeepLabV3+, and SegFormer. Based on the experimental results, semantic segmentation outperforms object detection regarding the use case, and the SegFormer architecture achieves the best results with a weighted MeanIOU of 0.853.</jats:p>