Waymo Introduces New Human Driver Benchmark to Measure Robotaxi Safety

Waymo has developed a new computer model intended to provide a more accurate answer to one of the most important questions facing autonomous vehicles: how does the performance of its self-driving system compare with that of a human driver?

The Alphabet-owned robotaxi company announced on Wednesday that it had published research on the new model in the journal Nature Communications. The system was developed in collaboration with researchers at TU Delft and is designed to improve the evaluation of autonomous driving performance against human behaviour.

According to Waymo, the new model represents a significant advancement over the simulation framework it has relied on for several years. It is built around a concept known as active inference. This theory suggests drivers continuously predict possible future outcomes and adjust their actions to achieve the safest and most predictable result.

Waymo says the updated framework will allow it to better understand how human drivers would behave in situations similar to those its robotaxi fleet encounters.

“For decades, the automotive industry has used physical and virtual crash dummies to evaluate a car’s safety features, including its hardware and structural integrity,” Waymo said in a blog post published Wednesday. The company added that the new model expands on that concept by acting as a behavioural benchmark for autonomous driving systems and representing reasonable expectations for how a careful and competent human driver would react during traffic conflicts.

Developing a realistic model of human driving behaviour has become increasingly important for autonomous vehicle companies seeking to assess and compare the safety performance of self-driving systems. The announcement also comes at a pivotal moment for Waymo as it expands operations into additional cities while facing growing attention from regulators and the public.

One recent example highlighted the importance of these comparisons. In January, a Waymo robotaxi struck a child near a school in Santa Monica, California. At the time, the company used its previous human-driver model to argue that an attentive human driver would likely have made contact at approximately 14 miles per hour. According to Waymo, the robotaxi reduced its speed from 17 miles per hour to 6 miles per hour before impact, and the child suffered minor injuries. The incident remains under investigation by both the National Highway Traffic Safety Administration and the National Transportation Safety Board.

Waymo says the most significant improvement in the new system, known as the Reference Driver, is its ability to reproduce driver behaviour before a collision occurs. Previous industry models, including Waymo’s earlier versions, were primarily focused on replicating last-second reactions immediately before impact.

The Reference Driver is designed to go further by modelling the decision-making process that unfolds in the moments leading up to a conflict.

According to Arkady Zgonnikov, an assistant professor at TU Delft, the model can simulate the internal sense of surprise a driver experiences in dangerous situations, creating a more realistic benchmark for autonomous driving systems than was previously possible to automate at scale.

Waymo also believes the model can be adapted for a broader range of driving behaviours beyond collision avoidance. The company says it is better suited for evaluating thousands of scenarios across large-scale testing environments.

“The model can represent and evaluate numerous complex, real-world crashes in a virtual environment, identifying performance improvements with unprecedented speed and efficiency,” Waymo said.

The company hopes the research community will help further develop the framework. To encourage collaboration, Waymo announced that it is making the model’s research code available under an academic, non-commercial licence. The licence permits use for scientific research, educational purposes, personal experimentation, and academic publication.

By opening access to the technology, Waymo aims to advance broader research into autonomous vehicle safety while establishing a more sophisticated benchmark for comparing robotaxi performance with that of human drivers.