Virtual Engineering Terminal –
Vehicle functions can be tested intuitively with the aid of Tangible User Interfaces (TUIs). The Audi model with the red markings on the touch-table display represents the car of the respective user. The rest represent other road users.
Piloted driving of the future
It’s not just game designers who work with virtual worlds these days. Audi also uses simulation technologies – when it comes to the development of electronic assistance systems, for instance. We meet at the Virtual Engineering Terminal at Audi Electronics Venture GmbH.
Andreas Kern steers his Audi A6 toward a wide intersection. A multistory building in gray concrete is blocking his view. Suddenly a red double arrow lights up in the cockpit – accompanied by an insistent tone. The Audi is warning the driver of potential danger. Although not yet visible to the human eye, the radar sensors have identified a car that is also approaching the intersection from behind the building. The Audi can begin braking in plenty of time and thus prevent an accident.
However, the development engineer for virtual environments at Audi Electronics Venture GmbH (AEV) is not really sitting at the wheel, but at what’s known as the Virtual Engineering Terminal. This is where Audi engineers demonstrate how piloted systems work. It also presents the opportunity to test new developments in prototype and virtual form. Right now, it’s working on the simulation of the intersection assistant.
Kern moves a model car on a horizontal touchscreen which presents a stylized version of the situation at the intersection and the sensor activity. When he moves the model, the representation also changes on a vertically mounted 65-inch monitor. It provides the user with a view of the virtual cockpit and a computer-generated landscape, which adapts precisely to the movements.
Ralph Stock watches the scenario with intense interest. The 47 year-old has been developing computer games for the past 30 years. When he now moves the Audi model across the gleaming surface, Kern pushes a miniature truck toward the simulated intersection. The virtual sensors react immediately. Kern comments on the benefits of development using virtual tools: “Things get tricky when you have several piloted cars in an intersection situation because I have to be able to accurately reproduce the test for a very specific constellation. That’s why we’ll be working a lot more in future with virtual worlds.”
And this is where the game developers come into play. Stock knows just how lively the dialogue between the different sectors has now become: “We’re so interesting to the automotive industry because we’ve been working with traffic simulations for such a long time.” They form the basis for many games and this experience is of huge interest when it comes to the high-precision simulations required by the automotive sector. The problem is that the networked systems are now far too complex to realize all the necessary tests with real cars. “Our aim is to reduce the number of real-life test drives, although we will still need them,” explains Kern.
And thus the collective knowledge of the gaming and automotive world are moving ever closer together – although the future reality of piloted driving will exceed the experience at the Virtual Engineering Terminal by orders of magnitude.
Expert discussion –
Audi development engineer Andreas Kern and game developer Ralph Stock use the Virtual Engineering Terminal to try out the intersection assistant currently under advance development.
Piloted parking –
The interactive installation also enables initial insights into highly complex new technologies like piloted parking in a suitably equipped parking garage.
As a development engineer with Audi Electronics Venture GmbH, Andreas Kern is working on computer-based systems for simulating virtual vehicle environments, of which the Virtual Engineering Terminal is part. The aim is to enable the analysis and validation of predictive functions in an integrated simulation of the vehicle and its environment.
Certainly when it comes to piloted driving, simulation is absolutely essential. It’s the only way we can test highly networked systems in a reproducible way. Real-life testing used to be pivotal. However, going forward, we’ll no longer be able to represent all situations in reality.
Development Engineer for Virtual Environments, Audi Electronics Venture GmbH
Audi is increasingly examining the development of new models and technologies in simulations similar to those used in computer games. We talk about the relationship between the virtual world and the real world – and how we can use it to steer the automotive future.
Mr. Kern, why do we need computer simulation to develop intelligent cars?
Kern: The enormous complexity of road traffic is particularly evident these days in cities. From a cultural perspective alone, European cities are completely different from South American or Chinese cities. To conduct real-life testing everywhere simply isn’t feasible. We have to develop safe, market-ready systems that function all over the world and with which the customer feels comfortable. We need simulation in order to do that.
So, without simulation, we wouldn’t be able to develop any new technologies at all?
Kern: Certainly when it comes to piloted driving, simulation is absolutely essential. It’s the only way we can test highly networked systems in a reproducible way. Real-life testing used to be pivotal. However, going forward, we’ll no longer be able to represent all situations in reality.
Mr. Stock, you’ve been developing computer games for thirty years. Where are the boundaries between simulation and gaming?
Stock: A simulation of real-life traffic forms the basis for many of our games. This means that, within our world, we successively construct the roads network, the traffic regulations and even pedestrian behavior. This means that passers-by might stop at a shop window, for instance. It all makes the world more realistic. What we end up with is a “world simulation” in real time. It’s not until then that we add the actual game with its rules. After all, players expect more than just a nice simulation; they want the full game experience – with points lists and progression options.
Are their similar procedures within the automotive industry, too?
Kern: To a certain extent, yes. As the development department for software methods, we serve as a resource to other departments when it comes to simulated Audi worlds. Our coworkers developing assistance systems come to us for the base technology that allows them to integrate their functions into the simulations. This enables the functions to be developed both in the actual car and in the simulation.
Why is the gaming industry so interesting to the automotive sector?
Stock: We’re of interest to other industries because we’ve been working on traffic simulation for years. It’s what makes our games believable and good to look at.
Kern: Yes, what we find fascinating about computer games is the physical accuracy of a simulated world. There are also similarities in the behavior models for the respective participants in simulations for the gaming and automotive industries. In a game, it’s the person who is generally set some kind of task. In the case of Audi, it’s the car that has to complete a task – such as warning the driver. And that’s exactly where it gets interesting, because within this context, the gaming industry has experience in how to correctly simulate pedestrian behavior, for example. We need this information, too.
And does that only work with external partners?
Kern: The automotive industry works with expensive, specialist software. The gaming industry, on the other hand, makes considerably more turnover with its technology and has also built up a great deal of expertise over the years. It makes sense to take a look at this sector and make use of synergies.
Does that mean making use of game engines?
Kern: Yes, that would be one example. We can use game engines to help us incorporate completely new applications into our development world.
What does that term actually mean in game development?
Stock: Generally speaking, a game engine refers to a graphics engine, i.e. a system that we use to make 3D worlds as realistic as possible. But there are other aspects – a physics engine, for instance, makes sure that gravity exists in the simulated world. Sound is another example. If I turn around, the source of the sound is now in front of me and not behind me. The sound engine therefore has to distribute the sound between the speakers in a way that makes it seem real. And then, of course, there’s artificial intelligence. We have a number of different requirements for that depending on the application. In the simulation of traffic flows, for instance, we need swarm intelligence, while in other products the focus is on simulating the reactions of individual people. This is how we use suitable engine components to build up the architecture of a simulation piece-by-piece.
And what do gamers learn from Audi?
Stock: There are some things we can learn when it comes to driving precision into the physical detail. For a game, it’s enough for a simulation to appear plausible for the game world. The guys from automotive design, however, have to reproduce reality because ultimately the car actually has to drive.
Kern: Beyond real-life test driving, what we need for networked and automated driving is a lot more test kilometers. This is something that can’t be reproduced in real life, which is why we have to make our simulations as precise as we possibly can.
How do the cars in the simulations know what they are doing?
Kern: We use a highly complex simulation engine that runs through certain scenarios to open standards. That’s how we define the screenplay. For us, it’s essential to define exactly what’s happening. We have to rely on that. And we also have to explain to the TÜV (German vehicle approval authorities) that we have developed the safety system in accordance with this method. But some things we intentionally allow to happen by accident – like in a computer game.
Does artificial intelligence play a role in that?
Kern: In our simulation, we have the option of saying to a “driver”: “You’re driving in accordance with the traffic laws.” The driver knows the laws and knows when he should brake. We also give these models characteristics like “maintain speed”. This means the driver in our simulation takes decisions with the aid of artificial intelligence on how to behave in traffic. In future, we will also transfer this capability into our piloted cars.
And how do you derive from the simulation what you need to know for the real world?
Kern: It doesn’t make any difference to the software whether it’s driving in a real Audi or in a simulated world. We use simulation to test and optimize vehicle functions as that presents no danger to anyone.
Stock: Plus, the simulation makes the results reproducible. In the best-case scenario – although we’re looking far into the future here – the software will learn within the simulation and optimize itself.
Are there developments within the automotive industry that wouldn’t be feasible without simulation?
Kern: That would mainly be swarm functions. We want to offer the customer added value by bringing to the road a fleet of vehicles that talk to one another, that aggregate and share information. That can only be tested using simulation.
So the topic for the future is swarm intelligence?
Stock: Absolutely! Increasing networking and the mixing of gaming and reality – like with Pokémon GO – is incredibly important to us. Reality is increasingly merging with the parallel data worlds. The challenge is to model it realistically. If we don’t consider relevant influences within the simulation – for instance, because we underestimate their significance – it can ultimately have a negative impact on the characteristics of the product being developed. That’s obviously why we have to be incredibly careful, especially when it comes to safety-relevant systems.
Kern: We’ll be much better in the city in future. We’ll need the right development methods if we’re to enable piloted driving in urban environments. Simulation with the behavior patterns of the different road users is utterly indispensible in that respect.
Sensor visualization –
The activity of the sensors necessary for piloted driving – such as pedestrian recognition – is recreated on the Virtual Engineering Terminal using intuitively comprehensible signs and symbols.
Ralph Stock developed his first computer game for the Commodore 64 back in 1984 when he was still at school – called Lapis Philosophorum: The Philosophers’ Stone, it was the first German game to achieve international success. Following many subsequent successes, he now manages Serious Games Solutions GmbH, which focuses on systems for use in fields such as health education and employee training.
A simulation of real-life traffic forms the basis for many of our games. This means that, within our world, we successively construct the roads network, the traffic regulations and even pedestrian behavior. What we end up with is a “world simulation” in real time.