Which factors might an autonomous car's crash behaviour take into account, and why is that list itself controversial?
Responsibility for the impending accident, the number of people endangered, their age, and whether occupants get priority — and each one, once programmed, becomes a standing rule about whose life counts for how much.
* Each answer, once programmed, becomes a standing rule about whose life counts for how much *
Because vehicles must be programmed in advance for certain reaction patterns, society first has to work out which procedure would be the most justifiable. The open questions are:
- Responsibility for the impending accident — should someone who caused the danger bear more of it?
- The number of endangered people — do five count more than one?
- Their age — may a car prefer the young?
- Occupant priority — is it legitimate for a vehicle to give the safety of its own occupants precedence?
What makes the list contentious is not any single entry but what a decision on it is. A driver who swerves in a panic makes no judgement about relative worth; a rule that prefers the young is exactly such a judgement, applied to everyone, forever, by people who will never meet those affected. Discrimination that would be intolerable if announced as policy does not become acceptable because it is expressed in code.
There is a necessary corrective, though, which the paper insists on: the thing at stake — the loss of human life — is not peculiar to autonomous vehicles. Road traffic already kills. Increasingly automated vehicles merely force us to look seriously at that dark side of our traffic system, and to say out loud what we have so far left to chance.
Go deeper:
MIT Moral Machine — judge the scenarios yourself, then see how your answers compare — the experiment that collected 40 million of them.
Wikipedia: Moral Machine — what the study found, and the serious criticism that it is no basis for policy.