IIHS Claims Waymos Are Safer Than Human Drivers If You Squint Real Hard While Reading The Data

Waymo robotaxis are either extremely dangerous or a "public health breakthrough," depending on who you ask. The Insurance Institute for Highway Safety has come out in favor of Waymo, declaring that its driverless cars crash less often than people. However, IIHS also freely admits having performed some statistical gymnastics to come to that conclusion, since the data to perform a direct comparison is hard to come by. 

It's easy to find data about crashes involving human drivers, but it's far more difficult to find comparable data for autonomous vehicles. While California has required autonomous crash reporting since 2014, and NHTSA has required the same reporting since 2021, the standards are different from those for humans. 

An important statistic in determining crash rates is total vehicle miles traveled. The Department of Transportation's report on Traffic Volume Trends lists annual vehicle miles traveled, going all the way back to 2000. Such mileage reporting is not required for autonomous vehicles, making it impossible to calculate a rate of crashes per mile traveled across the industry. Waymo voluntarily shares this data publicly, but other companies don't. That's why the IIHS study focuses on Waymo rather than Tesla, Zoox, or any other robotaxi company. 

One of the biggest problems with this claim is the size of the various data sets. IIHS had to do a great deal of work to "clean up" the data on autonomous vehicles from 2021 to 2024 before any comparison was even possible. That meant eliminating duplicate records, as well as "situations in which the automation wasn't engaged, the vehicle wasn't on a public road, or no real crash took place." Then it applied similar standards for reporting crashes to police as human drivers. "For example, if a person complained of pain but left the scene, the researchers coded that crash as unlikely to be reported to the police. If an airbag deployed, they coded that crash as police-reportable due to the higher impact speed and the cost of airbag replacement — regardless of other property damage or injury."

In the end, less than one-quarter of the autonomous crash data was even usable.

Using that common-sense standard, they estimated that only 22% of the 736 public-road crashes in which the automation was engaged were likely police-reportable — 89 crashes involving Waymo vehicles, 50 involving Cruise vehicles, 10 involving Zoox vehicles and 10 involving other companies. Because mileage numbers were only available for Waymo, they used its crashes to calculate self-driving vehicle crash rates. Of Waymo's 89 police-reportable crash involvements, 64 were in driverless operation, and the others involved supervised use of the automation.

After all this number crunching, here's the IIHS conclusion:

Overall, Waymo's crash involvement rate was 68% lower than that of human drivers — 76% lower in Phoenix, 35% lower in San Francisco, 71% lower in Los Angeles, and 4% higher in Austin, where the sample size was relatively small. Waymo vehicles were involved in 85% fewer single-vehicle crashes and 81% fewer injury crashes per [vehicle miles traveled] than human drivers.

How reliable are these statistics, though? 

According to NHTSA, there were 6,103,213 police-reported crashes across the US in 2021, and 5,930,496 in 2022. Add to that 6,138,474 crashes in 2023, and 6,180,241 in 2024 from another NHTSA report, and that's a total of 24,352,424 reported crashes involving human drivers, compared to just 89 for autonomous vehicles. The same disparity exists with the number of miles traveled. "Waymo vehicles traveled about 50 million miles in driverless operation over the study period, compared with about 222 billion miles by human drivers over the same period and in the same locations," says IIHS. 

It's hard to call this comparison statistically relevant. The autonomous vehicle sample size is orders of magnitude smaller than the number of crashes with human drivers. Also keep in mind that Waymo operates exclusively in big cities, where speeds are lower, and they don't drive on highways, skewing those numbers even further. I'm not saying IIHS is lying, and I don't believe that's their intention. But I do question how they can make a definitive statement that Waymos are safer than human drivers when their data on autonomous vehicles was so limited, and needed so much cleaning up that only 22% of it remained. Even IIHS conceded this point in the study's conclusion:

The method of coding narratives is unsustainable as these deployments and the number of incidents continue to grow. Practical application: This study identifies ways in which national crash and VMT data collection for L4 vehicles can be improved for more timely and accurate safety evaluations.