Enhancing Robotic Vision through Deep Learning Techniques: From Detection to Construction

This paper presents a robotic system developed to enhance automation in construction workflows through advanced AI and computer vision technologies. The system integrates a robotic arm with a 3D point cloud camera and state- of-the-art 2D pre-trained Deep Learning models, such as GroundingDINO and SegmentAnything, to detect and seg- ment construction elements in 3D dynamic, unstructured environments. By processing point cloud data from the cam- era and aligning it with real-world coordinates, the system achieves precise object localization, enabling tasks such as element handling and assembly. Designed to address chal- lenges like clutter, occlusion, and variability in construction sites, this system bridges the gap between controlled labora- tory conditions and real-world applications. Experimental evaluations highlight its potential to improve efficiency and adaptability in construction tasks.