OEM&Lieferant Ausgabe 2/2021

97 Driving at ARRK Engineering. “During the developing this visualization tool, we laid the foundation of a new evaluation method: the so-called Neurons’ Criticality Analysis.” Based on this principle, a reliable statement can be made about how important or harm- ful individual neurons are for correct object recognition. Interactive visualization of decision-making processes The interaction of the individual neurons in the numerous layers of a CNN is extreme- ly complex. Each layer and each neuron performs special tasks in the recognition of an object – for example, a rough sort- ing according to shapes or the filtering of certain colors. However, each step contrib- utes in a different manner to the success of correct object recognition and can, in some cases, even worsen the result. This complexity leads to the fact that the im- portance of individual neurons for the de- cision has been inscrutable so far. There- fore, ARRK Engineering has developed an interactive and user-friendly graphical interface to visualize these paths. “In this way, the decision-making process of a CNN can be visually represented,” Diviš said. “In addition, the relevance of certain neurons to the final decision can be increased, de- creased or even eliminated. In real time, the tool immediately determines the impact of these changed parameters after each ad- justment has been made. Thus, the impor- tance of certain neurons and their task can be more easily identified and understood.” The streaming of the data can be paused at any time for stress-free and convenient analysis. For this visual baking, the experts at ARRK Engineering chose the cross-platform pro- gramming interface OpenGL to ensure the greatest possible flexibility. This means that the software can be used universal- ly on any device – be it PC, cell phone or tablet. “We also placed particular empha- sis on optimizing the calculation and the subsequent graphical display,“ explains Diviš. “Therefore, frame rates (FPS) in par- ticular were checked in our final benchmark tests. Within this framework, we were able to determine that even when processing a video and using a webcam, the frame rate was stable at around 5 FPS – even when vi- sualizing 90 percent of all possible feature maps, which is roughly equivalent to 10,000 textures.” Analysis of the critical and anti-critical neurons For teaching the CNN within ARRK Engi- neering’s visualization tool, the deep learn- ing APIs TensorFlow and Keras are used as a basis, serving as a flexible implementation of all classes and functions in Python. Oth- er external libraries can also be easily con- nected. Once the neural network has been sufficiently trained, the analysis of critical and anti-critical neurons can begin – the Neurons’ Criticality Analysis. “For this, we offer modification such as addition of ran- dom noise, the removal of color filters, and the masking of certain user-defined areas,” Diviš explains. “Changing these values di- rectly shows how much individual neurons influence the decision in the end. It also re- veals which parts of the neural network may be interfering with the overall recognition process.” With the help of a sophisticated algorithm, the criticality of each individual neuron is automatically calculated. If the value of a neuron is above a certain level, it influ- ences the correct image recognition. The critical threshold can be adjusted as de- sired. “The final definition of this thresh- old depends on numerous factors – for example, functional safety requirements, but ethical aspects also play a role here and should not be underestimated,” Diviš adds. “Depending on what is desired, this value can be adjusted beyond that, ensur- ing the greatest possible flexibility of the tool.” In this way, the tool is not fixed to the current requirements and standards, but can be adapted to the new requirements at any time in the event of changes in safe- ty regulations. Increased safety through deeper understanding of CNNs With the visualization tool, ARRK Engineer- ing enables a graphical validation of neural networks. Thanks to the software in con- junction with the NCA, further steps can now reduce safety risks in autonomous driving through additional mechanisms for plausibility checks. “Our goal is to minimize or better partition the number of critical neu- rons so that we can rely on robust image recognition of the AI,” Diviš sums up. ARRK Engineering https://t1p.de/2zk9 Contact ARRK Engineering GmbH Frankfurter Ring 160 80807 Munich Phone +49 89 31857-0 info@arrk-engineering.com Share The interactive and user-friendly graphical interface is the heart of the tool. In this way, the decision-mak- ing process of a CNN can be clearly visualized. In real time, after each adjustment of the parameters, the tool determines their impact on the final result. Area 1 is used for the initial setting of the tool. Area 2 contains the pausable visualization of the interme- diate layers. Number 3 is the overview of CNN‘s layers and parameters. The area 4 allows the user to influence the input image. In area 5 you can see the weights of selected layers and additionally area 6 allows changes to the individual values. This image shows an example of the graphical representation of the weighting of a layer. The histogram can be visualized for each layer and additionally gives the mean value as well as its standard deviation. Graphics: © ARRK Engineering GmbH