Creating a Framework to Measure Safety in Autonomous Vehicles

January 15, 2019

As we discussed in a previous update, federal, state, and local agencies are struggling with the lack of uniform standards governing the development and testing of autonomous vehicles. A recent report prepared for Uber Advanced Technologies Group by RAND Corporation, Measuring Automated Vehicle Safety: Forging a Framework, attempted to create a framework measuring safety in Autonomous Vehicles (AV).

The report’s authors considered how to define safety for AVs, how to measure their safety, and how to communicate what is learned or understood about them. The framework proposed in the Report for AV safety has three components:

  1. Settings: contexts that give rise to safety measures, such as computer-based simulation, closed courses, public roads with a safety driver present or remotely available, and public roads without a safety driver.

  2. Stages: the life stages of AV models during which these measures can be generated. This typically involves a development stage, where the product is created and refined, and a deployment stage, where the product is released to the public.

  3. Measures: the meaning of new and traditional measures obtained in each setting as AVs move through each stage. One category of measurement consists of the standards, processes, procedures, and design requirements involved in creating the AV system hardware, software, and vehicle components. The other two categories are “leading” and “lagging” measures: leading measures reflect performance, activity, and prevention; lagging measures are observations of safety outcomes or harm.

However, a challenge to implementing this framework is the availability of statistically significant data. Currently, AVs are operating in small numbers and in limited situations. Further, a large amount of data related to AVs either is not publicly available or publicly accessible.

The report notes that certain categories of data, such as how an AV system perceives and interacts with the external environment, are unlikely to be shared between companies due to the highly proprietary nature of the data. Other categories, such as the external environmental encountered by the vehicle, could be shared via a database containing the environmental circumstances, infrastructure, and traffic, but that the data would need to be anonymized. The data could then be used in AV development and improvement. Existing traffic safety databases could also be updated to include more detailed data on AVs, and the anonymization and eventual analysis of such data will become more feasible as AVs become more common.

As described in the Eckert Seamans Autonomous Vehicle Legislative Survey, federal legislation (the SELF Drive Act in the House and the AV START Act in the Senate) was not enacted during 2018. The U.S. Department of Transportation issued Federal AV guidance in October 2018. The remaining legal regulations are determined at the state level. Uniform standards and increased information-sharing could lead to more reliability in measuring AV safety and greater predictability in the realm of product liability.

The report offers the following recommendations:

  • During AV development, regulators, and the public should focus their concerns on the public’s safety as opposed to the speed or progress of development.

  • Competitors should report on progress at key demonstration points and, to the extent possible, adopt common protocols to facilitate fair comparisons.

  • Safety events that occur in the absence of statistically significant data should be treated as case studies and used as opportunities for learning by industry professionals, policymakers, and the public.

  • Efforts should be made to develop a common approach specifying where, when, and under what circumstances an AV can operate. This would improve communication between consumers and regulators, and would make it easier to track and compare AVs through different phases of development.

  • Research should be done on how to measure and provide information on AV system safety when the system is frequently being updated. AV safety measures must balance reflecting the current system’s safety level with prior safety records.

As the RAND report notes, AV consortia have started to emerge, including the Self-Driving Coalition for Safer Streets, which was established by Ford, Lyft, Uber, Volvo Cars, and Waymo, and the Partnership for Transportation Innovation and Opportunity, whose members include Daimler, FedEx, Ford, Lyft, Toyota, Uber, and Waymo. These consortia are facilitating broad participation in standard-setting, and may eventually build momentum toward a larger degree of information-sharing about practices, tools, and even data. (See previous update: Automotive manufacturers, technology companies among those teaming up to PAVE the way for autonomous vehicle)

Studies and reports seem to be coming to a single conclusion: cooperation between policy makers, manufacturers, technology companies, and the public is a must. In the past, being first to market was a leading factor in progress. With AV technology, cooperation and sharing of information between interested parties, including the general public, seems to be the way to further the use of AV technology. This is still an area where there are many more questions than answers. Today, as demonstrated by the groups being formed, there is a willingness to work together in order to harvest the many potential benefits of AV technology for the good of all.

Click here to view a downloadable PDF of the legal update.

This Artificial Intelligence, Robotics & Autonomous Transportation Systems Legal Update is intended to keep readers current on developments in the law. It is not intended to be legal advice. If you have any questions, please contact a member of the practice group, or any other attorney at Eckert Seamans with whom you have been working.

Share This Post