Image simulation and virtual measurement planning

© Fraunhofer IPA

Image simulation

AI-based image processing is a key technology for digitization and automation in production. Image datasets required for visual inspection or object handling are often created using real setups with high manual effort. Additionally, many inspection tasks do not have sufficient images of defects, so balanced training datasets cannot be provided.

As a solution, synthetic images and 3D datasets can be generated. In this process, a virtual scene is created based on 3D models and sensor-realistic images are generated through simulation. Neural networks trained with synthetic data can also be applied reliably to real-world camera images in order to solve machine vision tasks.

Measurement planning

Determining optimal poses for measuring workpieces in 3D is a challenging and laborious task that often yields suboptimal results. By using reinforcement learning to plan 3D sensor measurements with the aid of models, optimal measurement poses for 3D measurements can be identified using CAD models of a wide variety of workpieces without having to perform real measurements.

Numerous CAD workpieces are required for training in order to derive effective measurement poses for a wide variety of component geometries. In addition to different 3D sensors, the optical features of the components are also taken into account in order to obtain complete and high-quality measurement data. Comprehensive measurement planning can be used to derive optimized strategies, e.g. in terms of component coverage.


Virtual measurement planning with reinforcement learning

Automated visual inspection systems are mainly configured by manual tests and by teaching of measurement poses. A new approach based on reinforcement learning is now being used to virtually determine suitable measurement poses for visually inspecting individual workpieces with a variable robot/sensor setup.


Handout Synthetic Data


Postcard Synthetic Data


Simulation of Defects


Object Handling using synthetic Data