Artificial Intelligence for Environment Detection

Gesture recognition for manual assembly processes

 

What's it good for?
In the MonSiKo research project, an assembly assistance system is being developed to train workers in manual assembly processes and detect process errors at an early stage.

What is new?
A 3D sensor is used to record the current scene, i.e. the assembly area with its respective objects and the worker’s hands. Machine learning methods are then used for segmentation purposes and to recognize hand poses and gestures. The methods compare the currently recorded workflow with the specified procedure and report any deviations.

Documentation of manual laboratory processes

 

What's it good for?
Manual laboratory processes are subject to strict guidelines, which above all require nearly every work step to be documented in detail.

What is new?
To facilitate this time-consuming and disruptive documentation obligation for laboratory technicians, processes are recorded and documented by a 3D sensor. Documentation takes place automatically thanks to machine learning methods for hand pose and gesture recognition, which can also be combined with speech recognition.

Early warning systems for cleanrooms

 

a)    Cleanroom Reliability Equipment Monitoring System (CREMS)

What's it good for?
If contamination is detected in a cleanroom, it is usually too late. With the "Cleanroom Reliability Equipment Monitoring System" (CREMS), researchers at Fraunhofer IPA are currently developing an early warning system: highly-sensitive sensors inside production facilities detect even minimal amounts of contamination (particles and/or chemical contamination) at the point of origin.

What is new?
This means that contamination can be detected at an early stage - long before it penetrates into highly-clean areas where it could cause damage. In addition, information about particle size ratios and concentrations, including time of occurrence, can be automatically forwarded to higher-level, IT-supported, analysis systems for evaluation. By using learning systems and specifically developed analysis algorithms, it is even possible to detect imminent wear (predictive maintenance) promptly.

 

b)    FlexNote

What's it good for?
If an employee notices that a machine is defective or damaged, a software-supported tool developed at Fraunhofer IPA called FlexNote documents all information quickly, transparently and completely digitally, and then forwards it to all the relevant people. In addition, a connector on the machine supplies the respective process data.

What is new?
Researchers are currently developing FlexNote further: in the future, depending on the situation, data from the controller will be collected over a longer period of time and analyzed by learning algorithms. On the one hand, this will result in an early warning system because the affected machine will report any faults or damage in advance itself. On the other hand, the tool will suggest recommendations for rectifying faults, which can be executed and documented via guided process flows.

Further Information

 

Reference project

Cleanroom Reliability Equipment Monitoring System (CREMS)

 

Reference project

FlexNote

 

Reference project

MonSiKo