Enhancing Robotic Vision through Deep Learning Techniques | From Detection to Construction

Projektleiter:

Digitales Bauen, Prof. Dr.-Ing. Nikolai Gerzen und
CAAD | Computer Aided Architectural Design, Prof. Dipl.-Ing. Hans Sachs

Forschungsschwerpunkt:

Digitales Bauen

Kurzbeschreibung:

Challenge: Construction sites are inherently unstructured, cluttered, dynamic, and unpredictable. Lighting varies, surfaces are reflective, and the scene changes constantly as work progresses. Standard robots cannot cope because they rely on fixed, pre-programmed environments where every object is in a known position.

Gap: Current robotic vision is split between two insufficient extremes. 2D models are powerful but lack depth, they cannot tell a robot where in 3D space to reach. 3D-native approaches require expensive domain-specific training data that does not exist for most construction objects. No existing system effectively bridges pre-trained 2D AI with the 3D spatial reasoning robots need on a real site.

Solution: We apply state-of-the-art 2D vision models directly within a 3D point cloud framework without retraining. A simple language prompt (e.g. "brick", "wooden block") is all that is needed to target a new object class.

Weitere Projektinformationen:

Enhancing Robotic Vision through Deep Learning Techniques: From Detection to Construction – The International Association for Automation and Robotics in Construction

ISARC 2025- Paper ID: 199: by Yusuf Aykin