Google has recently reported 314 total disengagements from autonomous mode, i.e. deactivations of the autonomous mode when a failure of the autonomous technology is detected or when the safe operation of the vehicle requires the driver to take immediate control of the vehicle. 37.9% disengagements were produced by perception discrepancy and 25.5% due to a software discrepancy.
In accordance with regulations issued by the California Department of Motor Vehicles (DMV), Google must periodically submit a report of disengagements from autonomous mode that have occurred when operating its self-driving cars on public roads in California. The DMV rule defines disengagements as deactivations of the autonomous mode in two situations: (1) “when a failure of the autonomous technology is detected,” or (2) “when the safe operation of the vehicle requires that the autonomous vehicle test driver disengage the autonomous mode and take immediate manual control of the vehicle.”
In the latest report, which covers the period from September 2014 to November 2015, Google claims it has operated its self-driving cars in autonomous mode for more than 1.3 million miles. Of those miles, 424,331 occurred on public roads in California during the period covered by the report. As part of testing, the company claims their cars switch in and out of autonomous mode many times a day. These disengagements number in the many thousands on an annual basis though the vast majority are considered routine and not related to safety.
During the reporting period, Google’s fleet of self-driving cars experienced 272 “immediate manual control” disengagements. This type of disengagements is produced when the software detects a technology “failure” – i.e. an issue with the autonomous technology that may affect the safe operation of the vehicle. In these cases, the test driver is given a distinct audio and visual signal, indicating that immediate takeover is required. Google claims their test drivers are trained and prepared for these events and the average driver response time of all measurable events was 0.84 seconds. Immediate manual control disengages are triggered primarily when the system detects a communication failure between the primary and secondary (back-up) self-driving systems (for example, a broken wire); when it detects anomalies in sensor readings related to our acceleration or position in the world (accelerometers or GPS); or when it detects anomalies in the monitoring of key functions like steering and braking. According to the report, the number of autonomous miles they are driving between immediate manual control disengagements is increasing steadily over time. The rate of this type of disengagement has dropped significantly from 785 miles per disengagement in the fourth quarter of 2014 to 5318 miles per disengagement in the fourth quarter of 2015.
In the reporting period, there were 69 events in which safe operation of the vehicle required disengagement by the driver. Of the 69 reportable safe operation events, 13 were “simulated contacts” – events in which, upon replaying the event in their simulator, they determined that the test driver prevented the vehicle from making contact with another object. The remaining 56 of the 69 events were safety-significant because, under simulation, they identified some aspect of the self-driving car’s behavior that could be a potential cause of contacts in other environments or situations if not addressed. In 10 of the 13 simulated contact events, the self-driving car’s predicted behavior would have, in simulation, caused contact (though 2 of these involved simulated contact with traffic cones). In 3 of the 13 occasions, a driver in another vehicle made a move that would have, in simulation, caused a contact with Google’s car.
Finally, the report also provides the breakdown of disengagements by cause. They claim they have used the causes mentioned in the DMV rule (weather conditions, road surface conditions, construction, emergencies, accidents or collisions), and additional causes labeled as unwanted maneuver, perception discrepancy, software discrepancy, hardware discrepancy, incorrect behavior prediction, or other road users behaving recklessly. Out of 314 total disengagements, 37.9% were produced by perception discrepancy (i.e. a situation in which the self-driving car’s sensors are not correctly perceiving an object) and 25.5% due to a software discrepancy (i.e. situations involving apparent software inadequacies that do not readily fall into other categories, such as map or calibration issues). According to the report, 89% of disengagements were produced when driving in city streets, rather than freeways, interstates or highways. The company claims city streets require the self-driving car to navigate complex road environments such as multi-lane intersections or unprotected left-hand turns, a larger variety of road users including cyclists and pedestrians, and more unpredictable behavior from other road users.