GAIA-3: Scaling World Models to Power Safety and Evaluation

Evaluating autonomous-driving systems at scale remains one of the defining challenges in advancing real-world autonomy. Real-world testing is essential for proving safety, but it is costly, logistically constrained, and increasingly data-inefficient. As driving models improve and make fewer observable mistakes, the number of miles needed for statistically meaningful conclusions rises sharply. Most of those miles are uneventful, offering little signal about rare, safety-critical behavior.

Simulation provides a path forward. Virtual environments can enable safe, reliable, repeatable, and scalable testing of driving models. Yet, despite their promise, existing simulation approaches fall short of enabling meaningful evaluation for modern end-to-end driving systems. Procedural simulators, the long-standing standard for autonomy testing, allow precise control but lack realism. 3D reconstruction-based simulators achieve greater realism but struggle with occlusions and dynamic agents. Our latest advances in world models deliver the best of both worlds: capturing the statics and dynamics of real environments while generating realistic counterfactuals that expand the coverage of real-world testing.

Wayve has been pioneering the use of world models to unlock a new paradigm in how autonomous vehicles are trained and evaluated. Building on our work in future prediction, dreaming about driving, predicting in a bird’s eye view, and learning a world model, we introduced GAIA-1 (Generative Artificial Intelligence for Autonomy) as our first step toward this vision. GAIA-1 demonstrated that generative models could learn from video, text, and action to produce realistic driving experiences. GAIA-2 expanded the horizon by introducing richer controllability, broader geographic coverage, and diverse vehicle embodiments through multi-camera, spatio-temporally coherent scene generation.

With GAIA-3, we take a bold step forward: transforming world modeling from a tool for visual synthesis into a foundation for autonomy evaluation. GAIA-3 generates driving scenes that are not only lifelike, but also structured and purposeful—designed to measure, compare, and accelerate progress toward safe, scalable autonomous driving.

GAIA-3 combines the realism of real-world data with the control of simulation. It allows us to take authentic driving sequences and re-drive them with precise, parameterized variations—for example, altering the ego vehicle’s trajectory while every other element in the scene remains perfectly consistent. Other agents keep their motion, lighting, and weather unchanged, and the world stays coherent. This ability to operate in a world-on-rails approach is a major step forward for generative world modeling, transforming evaluation from reactive measurement to proactive exploration of the edge cases that define safety.

The following example shows a continuous highway sequence generated with a programmatically controlled sine-wave camera motion, demonstrating GAIA-3’s ability to alter the ego vehicles’ trajectory, while maintaining all other scene elements coherently over an extended period of time.

The same methodology can be used to generate NCAP-style tests virtually and at scale, both within simulated test-track environments and across varied real-world conditions. While vehicle trajectories and timing remain consistent, the background, lighting, and scene dynamics change.

This capability unlocks systematic generation of crash and near-crash scenarios that can be evaluated using the same occupancy and trajectory metrics applied to real-world data—paving the way for scalable, reproducible safety validation.

A crucial benchmark for safety-critical generation is consistency: ensuring that when only the ego vehicle’s behavior changes, the rest of the scene remains physically and visually coherent. To validate this, we use real-world sequences captured with LiDAR and modify the ego vehicle’s trajectory to produce collisions with scene objects. We then align the LiDAR point clouds from the original recordings with the generated frames and check for misalignments and inconsistencies. Below, we showcase an example confirming that GAIA-3 preserves spatial structure and realism even when synthesizing safety-critical scenarios.

Consistency comparison:

Offline Evaluation Suites

Real-world driving rarely follows a script. Unexpected events, such as an abrupt stop, a late merge, or a pedestrian stepping into the road, reveal how well a model truly understands its environment. The ability to recreate and test these “what-if” moments offline is essential for building confidence in autonomous systems. With GAIA-3, this can be done systematically. 

By conditioning the ego vehicle’s behavior through action conditioning and optionally combining it with World-on-Rails perturbations, GAIA-3 can generate controlled variations of real-world scenarios. From a single recorded sequence, an entire family of “what-ifs” can be created by adjusting different parameters. These perturbations allow quantitative testing of a model’s ability to recover from edge cases or maintain stability under changing conditions.

The result is structured offline evaluation test suites that are scalable, repeatable, and measurable. They provide richer diagnostic signals than static replay, revealing how policy behavior shifts when conditions change. Correlation studies between GAIA-3’s synthetic interventions and on-road experiments indicate that the model can reliably predict relative policy performance, increasing the practical value of offline evaluation for model comparison and decision-making.

The examples below illustrate this: GAIA-3 is conditioned to drift to the left/right or go too fast/slow (trajectory in green), while the driving model should predict to bring the vehicle back to a safe position (trajectory in pink).

In the video above, GAIA-3 is used to create multiple four-example test cases of sub-optimal behaviour across five different scenes. In each, GAIA-3 drives to the left, right, too fast, and too slow. We see the worlds remain consistent, “on-rails,” providing useful counterfactual intervention states to evaluate a driving model’s ability to recover.

Embodiment Transfer

Different camera rigs and fields of view make it challenging to reuse data across vehicles. With embodiment transfer, GAIA-3 can re-render the same scene from a new sensor configuration, using only a small, unpaired sample from the target rig.

This means evaluation suites can be easily transferred across different embodiments or automotive OEM vehicle programs without requiring paired captures. The examples below show how GAIA-3 transfers a scene from one camera rig to another.

Original Embodiment: The original videos are captured from a vehicle fitted with a rig of 5 RGB cameras, one looking straight ahead, two looking towards the sides, and two more looking backwards.

Embodiment A: This vehicle is a different make and model from the Original Embodiment, but with a similar 5-camera configuration. Key differences are the panel that obscures the front forward camera view, the vehicle’s bonnet that appears flatter and shorter in the left and right forward cameras, and a more visible windscreen. GAIA-3 convincingly reproduces reflections from the scene, including those from oncoming vehicles.

Embodiment B: This is a similar vehicle to the Original Embodiment, but with a different camera rig. The front forward camera has a wider, more zoomed-out fisheye lens. The left and right forward cameras are angled further outward, and the rear cameras are angled further back and zoomed in.

Below are several examples demonstrating the consistency and realism achieved through this methodology.

Robustness and Interpretable Control

Driving models must remain reliable even when faced with changes in appearance, lighting, or semantics. However, these variations must be measurable to ensure meaningful evaluation.

We introduce controlled visual diversity, allowing the appearance of a scene to change while its underlying structure remains the same. This means elements like lighting, textures, and weather can vary, but the geometry and motion of the scene stay consistent. As a result, we can directly compare model performance across different visual conditions, evaluate robustness at scale, and better understand how specific visual changes affect model behavior.

Enrichment and Debugging

Rare failure modes are difficult to capture in real-world driving, limiting data coverage and slowing model iteration. GAIA-3 can learn from a small number of examples and generate structured variations around them, expanding scenario families such as braking or merging into rich, physically consistent test sets.

These rare events can be amplified into larger, labeled suites for focused testing or retraining, reducing the time between discovering a problem and verifying a fix.

The example below shows a specific behavior, harsh braking, generated as controlled out-of-distribution variations that help evaluate model behavior in situations that are difficult to reproduce through real-world testing. The same behavior can be translated across different environments and countries. For example, braking on a highway in the US, in an urban setting in Japan, or even stopping unexpectedly at a green light.

GAIA-3 can help expand datasets with rare out-of-distribution examples. Here, GAIA-3 can translate a specific failure mode—harsh braking in the middle of the street—into new scenes and geographies.

Improved Capabilities through Scale

Progress in world modeling is driven by scale: not only in parameters, but in data diversity, representational power, and the quality of generated experience. 

GAIA-3 is a 15-billion-parameter latent diffusion-based world model designed for scalable, realistic, and controllable offline evaluations for autonomous driving. To support this capacity, GAIA-3 was trained using five times more compute than GAIA-2 and on roughly ten times more data, spanning 9 countries across 3 continents. 

The dataset emphasizes safety-critical scene elements such as pedestrians, cyclists, signs, and traffic control infrastructure, ensuring that the model learns not just to imitate driving scenes but to understand and reproduce the elements most relevant to autonomous systems.

GAIA-3 yields clear improvements with sharper visuals of static and dynamic scene elements. Importantly, it also shows enhanced world modelling capabilities that maintain scene coherence over long trajectories and through moments of temporary occlusion. This example demonstrates the quality and consistency of the pedestrian in the rear camera through occlusion.

The following examples showcase GAIA-3’s ability to generate realistic pedestrians in motion, text signs that are clear and readable, important landmarks being generated faithfully and fine-grained details being preserved.