An explainable artificial intelligence (AI) is an AI whose inner method of function and decision-making are comprehensible. This makes it is possible, for example, to identify important features of input data for decision making, to understand the inner logic or, in the case of a specific decision, to find out why it was made in this way.
Many modern AI procedures are so-called black boxes. The need for explainability increases as the use of AI methods in key processes becomes more common. Explainability can help to better assess the working methods and limitations of AI models and thus to draw conclusions about the underlying processes and increase acceptance of AI by the workforce.
In addition, in order to assess the risk of productive use of AI methods, explainability is sometimes necessary or required by law, particularly in the case of regulatory and safety-critical applications.
Verifiable AI refers to methods which assess the safety aspects and robustness of an existing AI model. In most cases, mathematical methods are used; these ensure that a tested attribute of the AI can be evaluated with absolute certainty. For example, features such as the robustness of the AI with regard to malfunctions or the range of possible output results can be examined.
Verifiable AI plays a major role in applications that have to meet strict safety standards. Examples of this include collaborative robots, automated guided vehicles or intelligent machine tools. Clear norms or industry standards often apply, which the AI application used does not automatically comply with. In such cases, verifiable AI offers the possibility of testing and verifying appropriate boundary conditions. As a result, AI procedures can be used in processes that previously had to be performed manually or using inflexible rule-based systems due to strict regulatory requirements.
In many areas where artificial intelligence (AI) is used, accuracy and rigid decisions are often not sufficient. Uncertainty quantification plays a crucial role in making AI applications robust and reliable. The aim is to enable algorithms to use measurement uncertainties in processes or data directly, thus improving decision-making. In addition, planning and forecasting can be more effective and reliable in situations governed by inherent uncertainty.
Uncertainty quantification is primarily used in applications where safety is paramount, such as human-robot collaboration, but also in areas such as manufacturing, logistics or machine control, where errors often result in high costs. For instance, a robot in an assembly cell can provide preventive feedback if it is unable to fully understand the current situation, and the anomaly detection software in quality control can mark parts which have an anomaly detection result that is unclear. Uncertainty quantification in AI applications can thus considerably improve their suitability for use in practice.
Today, norms and standards assume a multitude of essential tasks. They support the development of safe products by reflecting the current state of research, therefore giving the customer faith in modern technologies. At the same time, norms and standards give manufacturers legal certainty when a new product is launched on the market. Ultimately, the same must apply for products that use artificial intelligence (AI). However, so far, a standard for intelligent machines and products does not exist.
Within the scope of the standardization roadmap of the German Institute for Standardization (DIN) and the expert council of the Association for Electrical, Electronic & Information Technologies (VDE), Fraunhofer IPA is laying the foundation for such a standard. In the process, it has become clear that artificial intelligence varies significantly from classical software and must therefore be given special consideration.
In particular, aspects such as determining the necessary degree of safety, the specifics of industrial automation and medicine, and ethical issues must be considered in relation to the need for safety, the ability to promote innovation and economic efficiency.
A further focus of the standardization roadmap is on how to determine and ultimately certify the quality of an intelligent system. Methods of Explainable AI (xAI), such as those being researched at the Fraunhofer Center for Cyber-Cognitive Intelligence, play a decisive role in this regard.