Prof. ZHENG Pai, Member of Research Institute for Advanced Manufacturing, Wong Tit Shing Young Scholar in Smart Robotics and Associate Professor of Department of Industrial and Systems Engineering, together with his research team, has developed the “Mutual Cognitive Human-Robot Collaboration Manufacturing System” (MC-HRCMS).
This innovative system enables real-time perception of complex environments, accurate interpretation of operators’ intentions, and skill transfer through simple demonstrations. It can autonomously generate process codes and execute high-precision tasks, and is already being applied in advanced manufacturing scenarios such as autonomous aircraft drilling and electric vehicle battery disassembly.
The MC-HRCMS utilises advanced machine learning and 3D scene perception to enhance efficiency, safety, and seamless human-robot interaction in complex scenarios. Through industry collaborations, the team has customised and deployed these systems for leading enterprises, addressing demanding requirements for precision and complexity.
To tackle challenges in semi-structured and unstructured production, the team introduced a novel vision-language-guided planning framework that integrates Large Language Models, Deep Reinforcement Learning, and Mixed-Reality Head-Mounted Displays. This allows robots to analyse visual and linguistic cues, understand complex tasks, and collaborate with humans more effectively. The head-mounted device offers real-time data and guidance, redefining human-machine interaction.
Press Release: https://polyu.hk/GaibX
| Research Units | Research Institute for Advanced Manufacturing |
|---|
You may also like
