Machine Learning Enhanced Sensor Systems

Fraunhofer EMFT develops and builds application-specific sensor systems and enhances their functionality with machine learning methods. Such "Machine Learning enhanced Sensor Systems" enable the realization of novel sensor solutions that save energy and bandwidth, shorten reaction times, and at the same time allow secure handling of sensitive sensor data.

Sensor Systems

For the sensor systems, both sensors developed at Fraunhofer EMFT and commercially available sensors are used. Sensor circuits with microcontrollers are designed for this purpose and the "embedded" software or firmware is programmed as required. These developments range from the AFE "Analog Front End" – the conversion into a digital signal via an ADC "Analog Digital Converter" – to the digital processing in the microcontroller and the various interface technologies, both wired (USB, RS232, RS485, OPC UA, CAN, Ethernet...) and "wireless" (BLE – Bluetooth Low Energy, 868MHz solutions, WLAN...).

In the field of sensor technology, complex optical analysis systems are designed to record absorption, chemiluminescence or fluorescence processes at different wavelengths or even an entire spectrum. This allows the analysis of material and surface properties as well as biological substances in agricultural and medical applications. Especially spectral evaluations in the VIS and IR range provide extensive information for a wide range of applications.

Machine Learning

"Machine Learning" is a subfield of Artificial Intelligence (AI) that uses statistical models based on training data to make certain statements. Thus, patterns and regularities in the learning data can be recognized and analyzed with unknown test data. With machine learning, for example, anomalies can be detected, states can be classified, but also a behavior in the future can be predicted. Of course, always with a calculated probability based on the experience and data from what has been learned.

In the area of sensor technology, Fraunhofer EMFT uses machine learning to process the collected sensor data, which is usually available as time series. This involves using "unsupervised" approaches for anomaly detection as well as "supervised" approaches for classification and pattern recognition for assignment to a specific state. In production engineering, for example, sensor systems with machine learning are used for "predictive maintenance". This predictive maintenance of gear oil etc. makes it possible to predict necessary plant maintenance before a failure is imminent. This allows expensive, regular maintenance work to be reduced to only what is needed.

Data Processing at the Edge

The use of "machine learning" methods directly at the sensor in microcontrollers paves the way for promising new solutions. So-called "Edge AI" or "Edge ML" can process sensor data locally in the sensor node. Local inference with ML models can be used to make statements directly at the sensor, which in turn can lead to further actions. All this saves energy and time, as the raw data does not have to be sent to the cloud for evaluation. This is particularly interesting for areas where sensitive data is processed, e.g. in medical technology. Even time-critical applications, such as hazard detection for future autonomous driving, are therefore possible directly in the sensor node, which would not be achievable with cloud-based processing. Another advantage lies in the more efficient IT infrastructure, which does not have to be expanded to enormous bandwidths in order to be able to guarantee the transmission of the raw data.