Intelligent Computer Vision for Digital Twin and Reinforcement Learning for Assembly Line Balancing

The Experiment, performed by INESC TEC, is labelled “Intelligent Computer Vision for Digital Twin and Reinforcement Learning for Assembly Line Balancing” and the aim was to achieve a balanced manufacturing line, where operator and equipment do match the production rate needed to achieve the target task time.

Purpose of the experiment was to introduce an Advanced Plant Model (APM) that allows to manage a digital twin of the shop floor, control robotic operations and receive from the robots information about the environment. The APM is an integration of the MES and of the digital twin. A first part of the experiment aimed to exploit AI to dynamically allocate production resources to manufacturing tasks: in case machines failures, operators unavailability and time variations the system is able to re-balance the manufacturing line, increasing its flexibility and efficiency. AI Reinforcement Learning has been used to assign the production resources to the tasks. To continuously adjust the running production schedule considering past delays and faults by reassigning operations to robotic mobile manipulators.

A second part of the experiment aimed to overcome the limitations of the digital twin, by combining the existing digital representation with the usage of computer vision integrated with artificial intelligence methods deployed either on the robot, the cloud, or in an edge layer. The experiments was based on camera installed on robots but also available in the plant. This allowed to achieve a system with more dynamism and reliability and in particular to be able to handle “new/unidentified” objects, not tracked previously. To integrate information collected through an IIoT system into a production planning software for real-time planning adjustments.