AI-based Object Position Estimation

In addition to intelligent image processing, this technology project for logistics also uses a roll-on gripper specially developed for gripping containers © Robomotion

But what if, in times of increasingly personalized production, the variety of parts increases? Often, these efforts become uneconomical. The same applies to the use of robots in warehouses, logistics centers, or in the retail trade: The variety of articles is extremely high and teaching in each individual object that a robot should be able to grasp is too costly.

In this context, AI technologies can make robots more autonomous and more flexible and enable new applications. Automation solutions become economically feasible even under challenging conditions. Researchers from Fraunhofer IPA are showing what such an example might look like in the deep picking project. Among other things, they are working with the company »Premium Robotics« and develop AI-based image processing solutions that enable robots to recognize and grip unknown packaging units on pallets.

In the past, classical image processing approaches that require a high degree of domain and expert knowledge were used for the localization and gripping of objects by robots. However, these methods are often not sufficient for more demanding tasks, such as gripping closely stacked products that all have a very similar appearance. In addition, the time-consuming training of new products is another obstacle. An experimental setup developed by Fraunhofer IPA and Premium Robotics illustrates the use case and benefits offered by AI. The closely stacked objects on the load carrier cannot be gripped or approached arbitrarily without affecting other objects. The task was to investigate different approaches for object detection and to test their application in the field of goods picking by robots. 

In addition, methods based on convolutional neural networks (CNN) for object recognition and pose estimation were investigated. Since training new products is correspondingly costly – training samples of image data have to be generated and manually annotated – the model training was performed in a simulation environment and the trained model was subsequently transferred to the real world. Successfully tested objects were all kinds of packages like open and closed boxes, trays with bottles, cans and cups, transparent objects, different types of packaging of any shape encountered in a warehouse and all of this under challenging illumination conditions and background reflections.