Enhancing Robotic Vision through Deep Learning Techniques: From Detection to Construction

This paper presents a robotic system developed to enhance automation in construction workflows through advanced AI and computer vision technologies. The system integrates a robotic arm with a 3D point cloud camera and state- of-the-art 2D pre-trained Deep Learning models, such as GroundingDINO and SegmentAnything, to detect and segment construction elements in 3D dynamic, unstructured environments. By processing point cloud data from the camera and aligning it with real-world coordinates, the system achieves precise object localization, enabling tasks such as element handling and assembly. Designed to address challenges like clutter, occlusion, and variability in construction sites, this system bridges the gap between controlled laboratory conditions and real-world applications. Experimental evaluations highlight its potential to improve efficiency and adaptability in construction tasks.