109 whereas in urban environments, potential obstacles may be located at a closer range. But are the ADAS of the different manufacturers even capable of doing this? To find out, the researchers focused on trajectory planning. For the deployment location Germany, they defined three driving scenarios with different speed limits and correspondingly varying requirements for the ACC: motorway (130 km/h recommended speed), country road (max. 100 km/h) and urban traffic (max. 50km/h). For their analysis, the researchers drew on a total of six data sets, including those that have been used by well-known OEMs for years: ONCE, nuScenes, A2D2, LyftLevel5, Waymo and Kitti. The first objective of the study was to determine the statistical distribution of the annotated objects that the training set contains. This involved assessing the size of the bounding boxes used for annotation, the relationship between their size and distance from each other, the distance between the vehicle and other objects as well as their relative position distribution on the sensor and the optical flow of the image sequences. With these parameters, the researchers determined how precisely the objects were annotated and accordingly tagged with bounding boxes in the first place. In addition, they investigated how well the camera sensors of the vehicles were adjusted to their operational areas of use and what proportion, for example, was accounted for by standing phases, during which (almost) static images were captured over a longer period of time. Optimizing the examined data sets The results of the analyses and the inferred quality of the data sets came as a surprise to the machine learning specialists at ARRK Engineering – and in a negative way. For example, they discovered an unexpectedly large number of static images from traffic congestion and idling, which were not marked as such and could therefore negatively influence the detectors’ accuracy. On top of that, all the data sets examined completely lacked annotated objects at greater distances of around 100 m and more. However, due to the short TTC at speeds of about 130 km/h, it is imperative that ADAS detect such distant obstacles in order to ensure safe use on German motorways. In addition, the annotation of the objects can often be inaccurate because many people are working on the data sets with different approaches. To compensate for this, the bounding boxes are set more amply than necessary and is not clear if they should overlap or not. This in turn makes it harder for the systems to detect obstacles and prolongs the training processes. The researchers now aim to improve the poor quality of the data sets with regard to the development of level 2 and 3 ADAS. Therefore, they developed an approach to validate the models considering the operational domains of the systems and to correct their deficiencies accordingly. For example, the precision and generalization of the detector trainings can be increased by eliminating images with ambigously overlapping bounding boxes as well as static images. In addition, the study results allow determining to which extent a data set is applicable to a specific operational scenario, and amending it accordingly during training. In the case of ACC, this includes driving in urban areas, in the countryside or on motorways. By better aligning the camera sensors to the actual traffic situation, the efficiency of the calculations and thus also the reaction time of the ACC can be improved. Increasing safety on the roads thanks to validated data sets Validating data sets with ARRK Engineering’s newly developed approaches allows reducing the required iteration loops in the training process and thus the overall development time of the ADAS significantly. More efficient training can therefore save valuable time already in the development phase. In addition, systems that have been precisely trained for their operational domain also feature higher functional safety. In practical use, for example, ACC can detect moving objects and stationary obstacles more reliably and initiate appropriate decelerating in time. Given that more and more highly functional ADAS will be on our roads in the future on the path to autonomous driving, this will increase general safety in daily road traffic. The paper was presented at the SafeAI 2023 conference https://safeai.webs.upv.es Lecture PDF https://ceur-ws.org/Vol-3381/33.pdf ARRK Engineering https://engineering.arrk.com The annotation of objects is often inaccurate because many people work on the datasets with different approaches. To compensate for this, the bounding boxes are set more amply than necessary and often overlap. Heat maps visualizing the distribution of the annotated objects over the entire sensor field of view reveal that 99.9 percent of the objects were detected in the lower half only. Image: © ARRK Engineering Image: © ARRK Engineering
RkJQdWJsaXNoZXIy MjUzMzQ=