Trolley Problem
Nick, Chris, Fraser and Peter discuss the trolley problem in relation to the terrorist attack in Stockholm.
For more information on Aleph Insights visit our website https://alephinsights.com or to get in touch about our podcast email podcast@alephinsights.com
Transcript
Hello, and welcome to the Cognitive Engineering Podcast produced by Tell Me Studios for Aleph Insights. In this series of podcasts, we take a look at interesting topics and discuss what we think they tell us about analysis and decision-making. I'm Fraser McGruer, and I'm here with Chris Wragg, Nick Hare, and Peter Coghill of Aleph Insights. And this week, we're discussing a real-life example of the trolley problem. So Nick, can you lead us in on this, please?
Speaker B:Yes, so there, as everyone will be aware, there was another terrorist attack in Stockholm recently, which involved a vehicle attack. And I think the thing that we're interested in is that a sort of man called Santiago Cueva, who happened to be sitting in his van at the time that the attack started, used his van to run the attacker off the road. So the attacker was in a lorry hurtling down the road, and Santiago Cueva basically pushed him into a department store. Now, as a result of that, some people got killed who would not otherwise have been killed. But he's being hailed as a hero for sort of taking his, putting his life at risk. And the argument being that he's probably saved a lot more lives as a result of having done that. But this is interesting because this is actually a real-life example of a really well-known moral thought experiment called the trolley problem, where a train is out of control and hurtling towards a group of five people who are, for some reason, having a picnic on the train tracks. And your choice is to flip this switch and divert the train down a different route. But on that route, there's a single person having, also having a picnic on the train tracks. And the question is whether or not you should flip the switch. So should you save those five people at the cost of one person who wouldn't otherwise have died? And, you know, so that's the question. And there's been all kinds of interesting variants of the trolley problem. But it forms a good basis for the discussion of, you know, what kinds of actions are right under various circumstances when you have to make these sorts of choices.
Speaker A:Okay, so a real-life example here, and the person involved took a decision and actually possibly didn't even take a decision because it was, as these things happen, it was in the moment and just went ahead with the course of action. And so having laid the foundations there, yeah, what are your thoughts? Anything immediately spring to mind from either Chris or Peter you wanna come in with here? Chris?
Speaker C:Yeah, I think for me, the issue of the trolley problem is that obviously it's got much more certainty attached to it than real-world problems. So, you know, it's simply a binary decision and the outcome, because it's a trolley, it's a, you know, the train is traveling along this thing, it can't go anywhere else. So there's a high degree of certainty about the problem. So you know that by acting in a particular way, the outcome is known. Where of course, in this example of the Stockholm attack, the outcome is much less certain. There are far more variables at play and there are far more outcomes than just, you know, two potential outcomes. And so I suppose for me, the issue why the problem is difficult and why most people seem to err on the side of inaction under these circumstances, I think is related to that uncertainty and people's judgment about the level of impact they will have on a situation and how likely they are to be successful. So I think most people on engaging in an action have a degree of optimism bias about their likely success. So if it's, I will commit an action that will kill one person versus some unknown outcome that might kill five people, but might not, then people will probably err on the side of, well, I'm not gonna do something which I know I'm gonna be successful at and kill one person when the outcome is uncertain. It might be five people die, it might be no people die. So I think that's, you know, that's where the thought experiment and the real world collide and why it's a very difficult dilemma and why often people do, you know, people do say, oh, it just happened, it was instinctive, you know, but under circumstances where people are able to calculate and have time to make that decision, I think they tend to err on the side of not doing the thing.
Speaker B:Yeah, people are, we know that people are loss averse, particularly when it comes to issues of, you know, with sort of saving lives and things that people would, people would generally take risks to avoid losses, you know, in a way that they wouldn't when you present those same losses as gains. I suppose the, you know, so we know that people do actually behave in ways that violate, you know, fairly sort of standard assumptions about how you should behave, where you should behave basically to maximize the total goodness in a sort of utilitarian setup. You shouldn't really care about how many people, you know, would die under one circumstance or another. It should be, or, you know, what the spread of outcome should be. Yeah, so we know that, you know, people do behave in ways that look sort of slightly irrational. The question is whether they should. So take the issue of uncertainty. Should it make a moral difference? Let's say if instead of, you know, there being exactly five, if we knew that there were between nought and 10 on one track and between nought and two on the other track, so on average, there were five on one and one on the other, you know, should that make a difference? So ought it to make a difference? But I know it does in real life, but should it?
Speaker A:Mm-hmm. Peter, anything you'd like to come in with here?
Speaker D:The trolley problem is very commonly cited and used when discussing automation and particularly used in discussing autonomous vehicles because it's quite probable that vehicles will face such decisions at some point that they have to take some action to either save the occupant of the vehicle or, and in doing so, potentially endanger individuals outside the vehicle. So it's a very relevant thought experiment today with the development in autonomous vehicles. That said, however, I think it's a bit over-talked about. I think a lot of the time, although it's likely this will happen, the actual number of occurrences will be vanishingly small because autonomous vehicles are generally engineered not to get themselves into a situation where they don't know what's going to happen with a high degree of certainty. So for example, if an autonomous vehicle is driving down the road and it knows there's a school nearby, it will slow down so that it has a greater horizon before it has to make a decision just in case a child crosses the road before it can stop. So to quote the Google engineers, they say, just don't hit stuff, period, is their approach to dealing with these situations.
Speaker B:But what about situations where the vehicle itself could be used to save lives? So if there's a lorry that's out of control hurtling towards a busload of nuns, and you could, the car, the autonomous vehicle could interpose itself and save the nuns, ought it to do so? That's the kind of situation we're talking about. And I think one of the things that Chris touched on there is this idea that there's a moral difference between acting and failing to act. So people often feel that if they act, they're somehow more liable than if they simply don't act or if they choose to do nothing and just stand there. And in this situation, I think people would probably resist the idea that their car could be used instrumentally to save third parties. But actually, in moral terms, there's no reason it shouldn't, you know, there's no, we should at least consider whether or not a car ought to sacrifice its occupant to save a bunch of other people elsewhere. So, you know, it's not just about the car not crashing, it's about the car being used to prevent negative outcomes.
Speaker D:I've got some data on that, actually. MIT did a consumer survey around autonomous vehicles and it turns out that most people would rather other people drive a very utilitarian vehicle, i.e. one that thinks about the best way of saving life, whereas they would personally very strongly not like to drive in one or be driven in one. So there's one rule for you and a rule for everybody else when it comes to autonomous vehicles and how they reason about saving lives.
Speaker C:Chris? Well, I think this gets to the heart of the issue of society, doesn't it? That actually, overall, what we're aiming to do is reduce everyone's risk. And so, yes, under circumstances, you think, hang on a minute, you know, if I'm in the car, I don't want to be saving the people at the bus stop. However, overall, there's a chance, there's a greater chance that you'd be one of the people at the bus stop and that somebody else's car would be, you know, used to save you. And there's something to me very equitable about the notion of pre-deciding that which you're able to do with autonomy and the programming for autonomy so that you all effectively sign a social contract by saying, okay, this is how vehicles will behave. And what you're overall doing is reducing the risk to any one individual in society, even if in one scenario, you're the one who bears the brunt of that.
Speaker D:And there's good precedence for that. That's the basic principle behind the highway code. You know, it's a social code of conduct that everyone, in order to drive, has to say that they're going to abide by. It's not a sort of law. It's a code. It's just sort of a set of principles that we all agree is the best way of going about stuff that can be challenged at any time by anybody and is often amended and revised.
Speaker A:But presumably with the highway code, the difference is, is that I can't think of an example. Maybe I can't think of that resulting in a, maybe it would, resulting in a death because of that code. Other, compared with an autonomous car, where by taking a certain decision, that will result in.
Speaker D:Well, an example could be minimum speed limits. So minimum speed limits on certain highways. You know, if the highway suddenly had more foot traffic or something, then the highway's minimum speed limit might be revised to a no minimum speed limit to reduce the risk. But until that was revised, there might be an increased chance of risk. So it's designed to take into account the limitations of human drivers, their reaction times and everything else. So, and minimize the overall risk to everybody, to all road users, as they call them.
Speaker C:But ultimately the difference here, I think, between, you know, having human agents in charge of the car and, you know, machine agents in charge of the car, is that under any given circumstance, you know, the selfishness of the human can override anything. You know, none of us would drive our own car in front of a bus stop. Well, very few of us would drive our own car in front of a bus stop to save a group of strangers at knowingly sacrificing our life. But a pre-programmed machine would not be considering it from our perspective. It would be considering its programming. And, you know, I think that's a different, do we all, it's almost like, you know, the concept of Odysseus being strapped to the mast to resist the siren's call. You know, if we all pre-accept a set of rules that we know we then can't override, I think that's an interesting societal issue that is different from the way we manage things.
Speaker B:Yeah, but this is, and this is the problem with considering it on a case-by-case basis versus instantiating it in a rule. And actually, this is where the trolley problem itself develops in an interesting way because people have developed other types of the trolley problem, where the decision is fundamentally the same, but where they seem to violate other sort of standards that we want. So one is the Southern Sheriff example, where, you know, there's a town where, you know, the townsfolk mistakenly believe that someone is guilty of a murder. The sheriff knows that he isn't, but if he hangs the innocent guy, he knows that he will stop a riot from happening. But if he doesn't, then there'll be a riot where, you know, a large number of people will be killed. And then you think, well, actually, they ought to be taken into account. If we ought to have a rule which says that, you know, you don't hang innocent people, in a way that having that rule is still, in net terms, better. And the other, another example, which is about a doctor who, you know, where someone comes in for a minor ear operation or something, and while they're under anesthetic, five victims of a car crash come in and by strange coincidence, all of them have a different organ injured. And by using the guy who's just come in for an ear operation as an organ donor, so by killing him, he can use all of his organs to save these five people who would otherwise die. Do we think that will be okay? Even people who are perfectly happy to pull the lever with the trolley problem, resist the idea that we should be, that doctors ought to be allowed to do that. And the reason is that because of the dynamic effects, ultimately, you know, it's the impact it has on behavior when you make it a rule. So if you say, look, actually, you can, you might accidentally be used as an organ donor if you go in for an ear operation, people won't do that. And I think, you know, if you think about the dynamic effects applied to things, you know, more real world trolley problem type situations, you know, if people know that cars are gonna swerve to avoid them, well, there's nothing sort of stopping them, you know, just blithely wandering across the road. And, you know, knowing full well that autonomous cars will kill their single occupants rather than kill, you know, say a group of five people crossing the road. That's the problem, is when people are able to adapt their behavior based on the existence of these rules, then you start to end up with actually quite difficult problems.
Speaker A:I want to come to Peter, but before I do, I mean, just sort of sum up some of what we've been saying, I think, slightly different to what you've just said then, but it just seems to me that we're in a new place at the moment where we've been, had these sort of issues for quite a long time or these examples of the trolley problem. But it seems to me that we're in a place right now because of increasing automation that for the first time or increasingly that you can apply logic more rigorously because of the automated nature of how technology is developing. I don't know if that makes sense, but, so in the case of an automated car, oh, sorry, an autonomous car, that just wasn't an issue before, but it is becoming, it is and is becoming more so. Is that a fair summation of, yeah? Okay, Peter.
Speaker D:I won't go into any more interesting examples of the trolley problem, save one. You can use it to explore whether or not direct agency and observation make a difference. So the fat man is an example where you can stop the train by pushing a fat man on the bridge onto the track. And that changes people's response because by physically being involved in the death rather than remotely via a lever makes a difference. Also makes a difference if the lever is obscured and none of the potential victims can see who diverged the train. So you can be an unseen agent. But so maybe going back to autonomous vehicles, maybe there needs some sort of random behavior generator needs to be included to mimic the way that humans behave in order to get around this gamification where people will jump out to cross the road at the expense of somebody's life because they know that it's gonna swerve to a void. So it obscures the thought process in the vehicles in much the same way that the thought process within our own minds is obscured from externals.
Speaker A:It just seems to me that there's a demand, there's a kind of emotional demand from humans to be able to do that, to slightly obscure the cold logic of it, which just feels slightly frightening.
Speaker D:So maybe when you turn on your vehicle in the morning, it picks a random point in a scale between very utilitarian, we'll try to save as many, minimize the risk from everyone, to totally self-preserving, and we'll do everything it can to preserve itself.
Speaker C:But also I think it's, particularly in the example of autonomous vehicles, it's an interesting angle from the point of view of optimization of the function of the vehicle, because risk is only one of the functions you're trying to achieve. And so, if you wanted to minimize risk, it would sit there in the garage and you're just sitting in the car. It'd turn itself off. It'd turn itself off, exactly. So- A strange game, the only way to win is not to play. Yeah, that's right. So you've obviously got the getting from A to B as quickly as you can, and as comfortably as you can, and all those other kinds of things, which are going to be weighed up to some extent. And there'll be trade-offs there as well, which we will have to somehow calculate. Is it reasonable for everybody to drive around at one mile per hour in order to avoid risk, or eliminate it as much as possible? Or do actually we accept that we all want to get to a place and we're prepared to trade lots of life against that? But I suppose societally, we do have speed limits, so we do make these decisions, and different countries have different speed limits to focus on that. So we are already making these decisions, but it feels more cold-blooded somehow.
Speaker A:Well, I want to come to you, Peter, and actually we need to round this off in a second. But actually what you were saying there reminds me of a Simpsons episode, where I think the national speed limit in the States is 60 miles per hour, or maybe- Could even be 55. Right, okay. Or it might depend on the state. Or it could depend on the state. But there's, for some reason, Homer Simpson is campaigning to raise the speed limit. And one of the things he says is that, sure, thousands of lives might be saved, but millions will be late, right? So you're basically on the same page as Homer Simpson, Chris. We need to round it up. So if anyone's got any comments they want to round off with, have a think about those. But I want to come to Peter.
Speaker D:From a practical engineering point of view, these occurrences are vanishingly rare. So where your car could potentially interdict the lorry is the minute number of these occur. So maybe it would be better to concentrate on cars with occupants being self-preserving and then have some kind of specialist robot, which is the lorry attacking pedestrians interdiction robot that hovered around shopping centers and did its job. So you specialize rather than trying to roll all of this risk consideration into one.
Speaker A:Sounds like a brave new world.
Speaker D:Yeah, but to take just one final point, to take the sort of Uber utilitarian point of view, these vehicles are going to do a far superior job to human drivers. Which will, the lives saved in reduced number of accidents will outweigh all those that are potentially lost through a vehicle that's a bit too vigilante or a bit too self-preserving. So, and every time a vehicle is in an incident, it will get better, all other vehicles will get better. It's not like drivers who learn from experience and that experience is held personal and can't be shared. Any vehicle will learn from the mistakes and the successes of other vehicles.
Speaker B:Yeah, and I mean, I suppose we can do the same thing in a moral sense. And if we discover that our cars are being too utilitarian and we don't like it, we can just change the slider somewhere and overnight all cars get updated to be slightly more selfish or whatever.
Speaker C:I mean, I can foresee a circumstance where effectively they have modes of driving. You can have the sort of road rage driver that's trying to get you to where you're going in a hurry. And that is your prime optimisation function at that point. And then you have the sort of driving Miss Daisy approach. And maybe you could indicate by the colour,
Speaker B:some sort of coloured lights, what mood that car is in. So if you see the red flashing lights, it shows that that guy, he doesn't give a shit about anyone and he'll happily run you over. So keep out of his way.
Speaker D:Well, that points to another little bit of data. So Uber and Google have totted up the number of accidents that their autonomous vehicles that they've been experimenting with have been involved in. And roughly it's twice the number of accidents per mile than human drivers, which sounds scary. But the analysis shows that most of the accidents weren't the fault of the car. It was other drivers not interpreting what this car was doing correctly because it was being too compliant with the rules. So it wasn't pulling out in front of traffic to get in the junction. So it was being rear ended and things like that. So actually, they've been involved in more accidents, but they've not caused them. They've been caused by the drivers not following the rules. So it was driving too perfectly in an imperfect world. It was driving too much like Miss Daisy. So people have suggested maybe you need like a rush hour mode, which gets into a bit more aggressive mode of...
Speaker A:I mean, we need to stop there, but just to round things off, it reminds me as it should in Aleph's style of other podcasts where we've had, where one of the things that I quite like this idea of having your car flashing amber and red if you're in road rage mode, because it reminds me of when we talked about irrational rationality, where it's great for that guy or girl driving that car because they'll get to work on time. And I wonder if you could mimic that and just have that going, even though you're driving like Miss Daisy and it will keep everyone off the road or out of your way.
Speaker B:Yeah, and social norms might develop that you're not really supposed to have your road rage mode on. And if you've got the red flashing light, people will shun you and you just have to take that into account.
Speaker D:Or maybe it costs, maybe you get a certain number of credits per month and you can use it just one time a month to get to your appointment because you're late or something.
Speaker A:Okay, look, we've got to wrap up there. But all I can say is Uber, you need to be listening to this. So we've got some people here can help you out with some stuff. Thanks as always for listening to the Cognitive Engineering Podcast. I'm Fraser McGruer. We've been here with Chris Wragg, Peter Coghill and Nick here at Aleph Insights. Thank you for listening until next time. Bye bye.
