Introducing Wayve Infinity Simulator

Introducing Wayve Infinity Simulator

Why is simulation important for autonomous driving?

Wayve aims to be the first to deploy autonomous driving technology in 100 cities worldwide. This depends on a vast amount of real-world testing to demonstrate that our AVs can safely and reliably drive through challenging urban environments. While real-world testing is a critical part of our development process, it also comes with some significant limitations:

• It is impossible to recreate the same scenario twice in the real world. Even on the same stretch of road, many things change between one test and another. This increases the amount of noise in the metrics.
• It is expensive in both time and money. Operational realities, such as the size of our development fleet and the number of AV Safety Operators, restrict the quantity of real-world testing that can be done.
• Some “edge cases” are too rare to observe in the real world or too risky to conduct tests on public roads, so we can’t rely on real-world testing alone to stress-test our system.

Simulation offers a powerful complement to real-world testing, addressing the above drawbacks while providing additional benefits. In simulation, we can recreate the same scenario as often as we like. Simulation is software and can thus be scaled arbitrarily and efficiently in the cloud. We can generate challenging, adversarial, or risky scenarios in simulation that would be unsafe to test in the real world and ensure our system can drive safely throughout.

Beyond testing, simulation offers us other tools to accelerate our development. We can explore how our driving intelligence behaves as we vary aspects of the scene, ensuring that it drives correctly regardless of weather, season, or time of day. We can rewind time and test a scenario repeatedly while varying the behaviour of the other agents in the scene. We can also use simulation to augment our training data sets to reduce bias and to unlock reinforcement learning approaches which depend on trial-and-error driving that would be impossible to conduct directly on public streets.

Wayve Infinity Simulator

Wayve Infinity Simulator is an end-to-end simulator custom designed to meet the needs of Wayve’s end-to-end driving system. We emulate the entire driving system, including the roads, traffic lights, behaviours of vehicles, cyclists, and pedestrians, vehicle cameras and other sensors, world physics, vehicle dynamics, and more. As a result, an AI model can be tested rigorously in simulation before being driven in the real world minutes later.

Infinity is built across four main pillars to ensure a high-quality simulation representative of the real world that we need to train our AI and evaluate its capabilities: realism, diversity, controllability, and scale.

Realism

Realism is a critical component of any simulation. Visual realism of the sensor simulation is essential, but it is only a small part of the story. Realism also critically encompasses the dynamic response of the vehicle to actuation commands, the latency and timing of execution on the virtual car, and much more. We’ll return to this topic in more detail in a future blog post.

Diversity

Realism alone is not particularly useful: imagine a perfectly realistic simulation of a single straight road devoid of other vehicles or pedestrians. This would not take us very far. So beyond realism, we must also have diversity that exposes our driving intelligence to as broad a range of driving scenarios as possible so it can learn to navigate the real world in all its complexity. This requires a system that can not only produce the full range of scenarios we see in the real world but also generate vast numbers of variations of each.

Trying to handcraft individual scenarios to cover this rich diversity would be impossible. Instead, Infinity’s procedural generation systems capture the characteristics of the real world as parameterised rules, allowing us to generate countless real world-like variations at the touch of a button. We can randomise every aspect of our simulated scenarios: the roads, building types, signals, weather, time of day, and the behaviour of other road users. Everything is parameterised to provide as much diversity as possible.

Controllability

At times we may want to create highly constrained scenarios to target specific areas of interest to our driving intelligence. These might be particularly challenging scenarios or safety-critical edge cases that would be impossible to engineer in the real world.

Infinity gives us complete control over the simulated environment. Every aspect of the simulation is configurable and can be set to a targeted value or randomised depending on our requirements. The simulator is fully scriptable, meaning a few lines of code can control any aspect of the simulation even while it is running. Further, we can easily access as many sources of ground truth labels, such as depth or semantic segmentation, as we need.

We’re just getting started

As we continue to develop Infinity, we’re iteratively enhancing the system across all four pillars of realism, diversity, controllability, and scale to constantly improve the quality of the insights we get. That, in turn, helps accelerate the development of our AV2.0 driving intelligence. While there is plenty we are still working on, we already see enormous value from simulation propelling us forward at an accelerating pace. In particular simulated, off-the-road evaluation allows us to efficiently build the scaling curves we need to scale up our AI foundation models for autonomous driving.

As with many aspects of the AV space, simulation is a rapidly developing field. Despite all the benefits simulation provides us today, we have barely scratched the surface of the enormous benefits it can afford. With the fully controllable, scalable end-to-end simulator we have built, we have the perfect platform to iterate towards AV2.0 quickly, efficiently, and safely.