The Cognitive Neuroinformatics research group at the University of Bremen has contributed important research successes to the development of advanced driving assistance systems in a cooperation project with the automotive supplier Continental. With the help of artificial intelligence, complex traffic situations can be better recognised.
Proreta 5 is the name of the research project that the automotive supplier recently completed with its scientific cooperation partners. In addition to the University of Bremen, the TU Darmstadt and the TU Iași (Romania) were also involved. “In the end, there was a driving demonstration in Darmstadt. There we presented autonomous driving functions that we had worked on intensively,” says Professor Kerstin Schill, head of the Cognitive Neuroinformatics working group at the University of Bremen. The research vehicle was able to autonomously follow the course of the road with a predefined destination and react to other road users – pedestrians, cyclists and other vehicles, explains the researcher.
The goal of the fifth Proreta research project was to develop algorithms that would derive correct driving decisions comparable to those of humans from sensor data. At an uncontrolled intersection, for example, it is already a challenge for human drivers, and even more so for control electronics, to interpret all objects relevant to the planned direction of travel. In the process, the driver’s own direction of movement as well as that of other objects, intention and priority in the traffic are included in the calculation. The artificial intelligence (AI) should be able to make safe decisions without human intervention. “The great advantage of AI is that after a training phase, it is able to draw the right conclusions in unknown situations based on what it has learned,” explains the computer science professor. “One part of the project was to observe the human drivers as they reduce and evaluate the complexity of the environment themselves. The adaptive algorithms are now being trained according to similar principles.”
In the project, the Cognitive Neuroinformatics working group investigated AI methods for environment perception – objects and obstacles should be recognised in the environment. In addition, new methods for human attention modelling were developed based on camera data. This involves creating conspicuity maps that determine relevant areas in the image where, for example, other road users or signs appear. In addition, new mathematical models have been developed that mathematically correctly represent the position, orientation, speed or size of other road users and describe complex vehicle geometries.
Tasks are now solved more efficiently, robustly and safely
Lastly, object tracking was implemented, which can perceive road users in the monitoring area and estimate their condition over time. “These methods ensure that the corresponding tasks can be solved more efficiently, robustly and safely. They thus make an important contribution to highly automated and autonomous driving,” says Schill. “The project is an ideal example of how profitable cooperation between university and business research can work.”
As part of the project, several researchers explored specific sub-aspects of applying AI to driving a vehicle. Jaime Maldonado worked on human attention modelling in the context of autonomous driving. In particular, an attention-driven pipeline was developed that consists of two components. On the one hand, relevant areas in camera images are determined by means of so-called saliency maps. On the other hand, the driver’s gaze is projected into the image to expand the relevant area. This allows relevant and non-relevant regions in the image to be distinguished and processed more efficiently by subsequent algorithms.
Andreas Serov implemented object tracking that perceives relevant objects in the vehicle’s surveillance area and determines their position, speed, orientation and size in real time. A list of tracked objects is made available to the following modules (prediction, planning and control) for further processing. Object tracking is based on radar and lidar data. The state of each object is estimated with a probabilistic filter, processing the state on a manifold.
Lino Giefer investigated theoretical foundations for state estimation and representation in autonomous driving. In particular, he established new models to describe articulated vehicles – such as buses, trams or vehicles with trailers – in a mathematically correct way. He also investigated state and measurement uncertainties for localisation and object tracking.
Razieh Khamseh-Ashari explored multimodal object detection based on lidar and camera data using AI methods. By early fusion of sensor inputs, a highly precise localisation of objects in the surveillance area is achieved.
The automotive supplier Continental was involved in the project as a partner. The company provided a vehicle equipped with sensors and a high-performance computer, which enabled the researchers to test the developed functional and verification methods for the automated driving system under real conditions. The methods included multimodal prediction of dynamic behaviour of an object, specifying and testing compliance with traffic rules, and logic-based testing to detect unsafe behaviour of AI modules.
Andree Hohm, PhD student at Proreta 2 and now head of the “Driverless” innovation area in the Autonomous Mobility business unit at Continental, comments on the e-results: “We conducted research at Proreta in order to combine knowledge in industry with the skills of university research and to find solutions for highly challenging problems together with young scientists. What is fascinating about this cooperation is that we have used answers from research to lay the foundations for actual application in the vehicle. What we developed in the first projects can be seen in road traffic today and ensures safety on our roads every day.”
The first Proreta project (2002 to 2006) investigated how a vehicle can automatically detect imminent dangers in the form of stationary or encroaching obstacles by means of environmental sensor technology and avoid them by emergency braking or emergency evasive action. In the Proreta 2 project (2006 to 2009), the teams researched the prototypes of a driver assistance system that helps the driver to prevent accidents during overtaking manoeuvres on rural roads. To this end, a system was developed that uses sensor and driving dynamics data to determine and permanently calculate the position of the driver’s own vehicle, the vehicle in front and any oncoming vehicle in order to determine whether the clear path is sufficient for a safe overtaking manoeuvre.
Proreta 3 (2011 to 2014) researched the implementation of a “safety corridor” as an integral vehicle safety concept to increase active safety in urban and rural road traffic. Challenges here were the high complexity of urban traffic and the question of how a safety system must be designed that recognises a sudden danger and takes active assistance measures such as steering and/or braking interventions so that it is accepted by the driver. Proreta 4 (2015 to 2018) focused on intelligent learning vehicle systems to further increase driving safety and comfort. These assistance systems support the driver in difficult situations with individual and adaptive recommendations