Projects

IMAGE RECOGNITION AI FOR ROBOT OPERATION MONITORING AND ACCIDENT PREVENTION

14 September 2021 / Last updated : 14 September 2021 araya AI Case Study

The use of AI and IoT to automate processes is rapidly being adopted in a variety of industries.
In the manufacturing industry, the use of AI, IoT and robots is expected to significantly increase productivity. Factories are becoming "Smart Factories", where data visualisation and automation of equipment and machinery in the factory is starting to take place, and various types of robots are also being used. The robots used in factories include industrial robots, AGVs (Automated Guided Vehicles) and AMRs (Autonomous Manipulator Robots).
The transport, retail, distribution and service industries are also beginning to use transport robots, food delivery robots and service robots such as Pepper.
Furthermore, robots are expected to be used in the "smart city" concept of the near future. A smart city is an initiative to improve people's quality of life by using ICT, AI and other advanced technologies to automate and optimise all aspects of the city's environment, services, management and operations.
Improving the operation of these robots will directly lead to increased operational efficiency in the field in various industries.

While there are great benefits to the use of these robots, there are also some challenges, such as the fact that many accidents involving people occur when robots and people work together.

AGVS (AUTOMATED GUIDED VEHICLES) AND AMRS (AUTONOMOUS MANUALLY OPERATED ROBOTS) ARE INCREASINGLY BEING USED IN FACTORIES AND WAREHOUSES TO REPLACE CONVENTIONAL TROLLEYS AND CONVEYOR BELTS. AN AGV IS A VEHICLE THAT AUTOMATICALLY TRAVELS ALONG A FIXED ROUTE, FOLLOWING MAGNETIC TAPE OR OTHER DERIVATIVES AFFIXED TO THE GROUND, AND IS CAPABLE OF TRANSPORTING PRODUCTS OR PARTS UNATTENDED. AN AMR IS A ROBOT THAT AUTOMATICALLY CALCULATES A ROUTE AND TRANSPORTS OBJECTS WITHOUT THE NEED FOR MAGNETIC TAPE.

THESE AGVS AND AMRS POSE THE FOLLOWING OPERATIONAL CHALLENGES

Issue 1: Contact and collision accidents with robots

Safety measures are important for AGVs and AMRs because of the possibility of contact or collision with workers or obstacles. Of course, they have built-in control systems, which can stop the vehicle if it detects an obstacle or a person, but these systems can only function at the last minute and cannot cope with quick movements.
Sudden stops and starts can also lead to loads collapsing or falling during transport.

Challenge 2: Inefficiency due to emphasis on secure operations

ON THE OTHER HAND, TOO MUCH EMPHASIS ON SAFETY CAN LEAD TO SIGNIFICANT WORK LOSSES, DUE TO THE SLOWING DOWN OF THE ROBOTS AND THE MANY TIMES THEY HAVE TO STOP TO AVOID DANGER. SUCH INEFFICIENCIES ARE ALSO CAUSED BY CONGESTION CAUSED BY THE INCREASING NUMBER OF AGVS INSTALLED.

BY INTRODUCING THE ROBOT OPERATION MONITORING SYSTEM, WHICH BRINGS TOGETHER ALAYA'S IMAGE RECOGNITION AI TECHNOLOGY, THESE ISSUES CAN BE ADDRESSED TO PREVENT ACCIDENTS CAUSED BY ROBOTS AND ENABLE ROBOTS TO OPERATE MORE EFFICIENTLY.

Features of the robot operation monitoring system

The functions and features of ALAYA's robot operation monitoring systems include the following

Prevention of contact, collision and deviation of robots

FROM A FIXED-POINT CAMERA, THE ROBOT'S AND HUMAN'S MOVEMENTS ARE CONSTANTLY MONITORED. IF THE AI DETECTS THAT THE ROBOT IS ABOUT TO COME INTO CONTACT WITH A PERSON OR AN OBSTACLE, IT CAN INSTRUCT THE ROBOT TO REDUCE SPEED OR STOP. IT CAN ALSO DETECT WHEN THE ROBOT HAS DEVIATED FROM ITS ORIGINAL ROUTE AND INSTRUCT IT TO STOP.

Multiple cameras can be linked together to cover large sites and complex areas with many blind spots

Multiple cameras can be linked, so that if a robot leaves the field of view of one camera and enters the field of view of the next camera, it is identified and tracked as the same robot/person. This means that the system can be used in factories that are too large to be monitored by a single camera, or in warehouses with many blind spots due to shelves, etc.

More cooperation between people and robots

Because it keeps track of the movements of both the person and the robot, it is possible for a transport robot to automatically pick up a product when it detects that the person is about to complete an assembly task, for example. This reduces the time lost in the manufacturing process.

How the robot operation monitoring system works

Find out how ALAYA's robot operation monitoring system works.
First, we use surveillance cameras on the premises where the robots will be operating. If there are any blind spots, we will install additional fixed-point cameras.
The image recognition AI will use the information from each camera to identify the robot and the person and locate them on the map. By keeping track of each position in succession, the direction and speed of the robot or person can be monitored. This allows us to detect at an early stage any deviation from the robot's route or any risk of collision with obstacles or workers.
The results of these AI predictions are then linked to the robot and control equipment to avoid contact and collision accidents while limiting excessive stopping.

Demonstration video: Predicting collisions between robots and people

The video below shows a demonstration of human-robot collision prediction.
The squares on the human and the robot show how the AI detects each of them.
On the map at the bottom right, the position, direction and speed of each robot are shown as vectors.
If the AI detects that the robot and person are about to come into contact with each other (in the demo video, the frame of the person who is about to come into contact with the robot turns red), it can send instructions to the robot to reduce speed or stop.
In factories and warehouses, the system can predict contact between AGVs (Automated Guided Vehicles), AMRs (Autonomous Manually Operated Robots) or other vehicles and people, and warn with an alarm to avoid the risk of an accident.