Learning to drive like a human

The solution is machine learning, which is surpassing hand-engineered systems everywhere. Intelligent behaviour cannot be hand-coded, but can be learned through experience. We’ve built a system which can drive like a human, using only cameras and a sat-nav. This is only possible with end-to-end machine learning.

• No HD-Maps,
• No expensive sensor/compute suite,
• No hand-coded rules,
• Driving on roads never-seen during training.

This scales self-driving technology like never before: for everyone, everywhere.

In this video you can see our system driving on public roads in Cambridge, UK. It’s driving on roads it has never been on before using just a simple sat-nav route map and basic cameras. We don’t tell the car how to drive with hand coded rules: everything is learned from data. This allows us to navigate complex, narrow urban European streets for the first time.

End-to-end deep learning

Why is our technology different? It learns end-to-end with imitation learning and reinforcement learning to drive like a human, using computer vision to follow a route. Imitation learning allows us to copy behaviours of expert human drivers. Reinforcement learning lets us learn from each safety driver intervention to improve our driving policy.

Our model learns both lateral and longitudinal control (steering and acceleration) of the vehicle with end-to-end deep learning. We propagate uncertainty throughout the model. This allows us to learn features from the input data which are most relevant for control, making computation very efficient. In fact, everything operates on the equivalent of a modern laptop computer. This massively reduces our sensor & compute cost (and power requirements) to less than 10% of traditional approaches.

Here are some videos of our method, demonstrating driving many kilometers on roads not seen during training, with complex, narrow and challenging urban situations with varying weathers in Cambridge, UK. All videos are in real-time and are not sped up. At all times, trained safety drivers ensure the AI system obeys the road code safely.

Here’s a quick example of our model demonstrating complex behaviour, navigating a narrow British street and then giving way to a passing car at a T-intersection.

But this longer demonstration is more interesting: it shows our vehicle driving for an extended period of time, completely autonomously, and on roads never-seen-before during training. What is more remarkable, is that this model learned this driving behaviour with only 20 hours of training data. We didn’t tell it to drive on the left or to slow down for give-way intersections. Every aspect of its behaviour is learned from patterns in the data.

Look closely in the video for the sat-nav that the algorithm uses to navigate, which uses only a consumer-grade GPS. The car navigates in cyclist and vehicle traffic, in the rain.

Here’s a final quick example of our model driving through a traffic light intersection.

The autonomous vehicles journey is just beginning…

Over the last year, we’ve shown three world firsts on autonomous vehicles with machine learning:

• using reinforcement learning to learn to drive from scratch
• transferring a model which learned to drive in simulation, to the real world
• navigating around never-seen-before complex urban environments using just a sat-nav and a camera sensor (this work).

The future is electric

We’re excited to announce that our autonomy platform is built on the Jaguar I-PACE. This vehicle won the 2019 European Car of the Year. The I-PACE is a fully-electric SUV vehicle and we’re proud to be championing technology that is clean and sustainable.

In the next year, we’ll continue to see our growing fleet of Jaguar I-PACE vehicles testing algorithms and collecting data throughout the UK and mainland Europe.

With each safety-driver intervention, our system learns and will improve, rather than buckle with scale. It will take us longer to reach our first deployment, but we are riding a fundamentally different curve.

We’re going to be the first to deploy autonomous vehicles in 100 cities.

With special thanks

To Admiral for insuring our autonomous vehicle trials.