An autonomous driving dataset made for novel view synthesis

Benchmarking Novel View Synthesis Models for Driving

Our publicly available WayveScenes101 Dataset provides 101 scenes from a variety of driving environments, such as urban, suburban, and highways, across different weather and lighting conditions. Each scene includes time-synchronised views from five vehicle-mounted cameras and camera poses.

We aim for this benchmark to advance state-of-the-art novel view synthesis models and support the development of more robust and accurate scene representation models for driving.

Get started now

Dataset Features

Our dataset uniquely targets applications of novel view synthesis for driving scenarios and offers features which are not commonly found in existing datasets.

We provide an evaluation protocol for a held-out evaluation camera to specifically measure off-axis reconstruction quality, which is crucial for estimating the generalisation capabilities of novel view synthesis models. The dataset features scenes across diverse locations, environmental conditions, and driving situations. A frame rate of 10 frames per second for each camera allows for accurate scene reconstruction, in particular for dynamic scenes. Furthermore, we provide metadata for each scene to allow a detailed breakdown of model performance for specific scenarios, such as nighttime or rain.

To summarise, the key features are:

  • 101 highly diverse driving scenarios of 20 seconds each
  • 101,000 images (101 scenes × 5 cameras × 20 seconds × 10 frames per second)
  • Scene recording locations: US and UK
  • Five time-synchronised cameras
  • Held-out evaluation camera for off-axis reconstruction measurement
  • Scene metadata for nuanced per-scene model evaluations
  • Integration with the NerfStudio framework
  • Out-of-the-box compatibility with NerfStudio

Scene Diversity

Our scenarios encompass a broad spectrum of environmental conditions and driving situations. These scenes were selected to be representative of the challenges faced when deploying novel view synthesis methods in the real world, such as dynamically changing traffic lights or moving pedestrians. We also provide detailed scene metadata for nuanced evaluation of novel view synthesis models, i.e. the presence of rain, the dynamics of the scene, and the time of day. Below, we list some of the key scene features:

  • Weather: Sunny, cloudy, overcast, rain
  • Road Types: Highway, urban, residential, rural, roadworks
  • Time of Day: Daytime, low sun, nighttime
  • Dynamic Agents: Vehicles, pedestrians, cyclists, animals
  • Dynamic Illumination: Traffic lights, exposure changes, lens flare, reflections

Camera Setup

Our camera rig includes 5 cameras that capture a wide view around the vehicle. Each camera features high-resolution sensors with 1920×1080 pixel imagery.

We supply COLMAP camera poses for all images along with the reconstructed 3D points. For off-axis reconstruction assessments, we use a separate held-out evaluation camera—the central forward-facing camera (grey frustum)—while the other four cameras (blue frustums) are designated for training.

Github Repository

Getting Started

To use our dataset and benchmark, visit our GitHub repository for download instructions and additional information.

Download Dataset

Interactive Demo Scene

We visualise the COLMAP points and camera poses.

Terms of Use

We provide the WayveScenes101 Dataset for non-commercial, research purposes only. By downloading the dataset, you agree to the terms of this licence.

Read our latest research