Irrationality
In this episode, we begin with Hitler’s supposed Alpine Redoubt and use it to explore a broader question: how do we make sense of irrational decision-making? We discuss why Allied planners expected the Nazis to behave “sensibly” by retreating to Bavaria, when Hitler instead chose a symbolic last stand in Berlin. From there, we look at other examples where leaders appear to act against their own interests, including Putin’s invasion of Ukraine and Trump’s strike on Iran.
We then set out a framework for irrationality, asking whether bad decisions come from strange goals, wanting the wrong thing, misunderstanding the world, choosing tactics that do not work or simply failing to think clearly. We consider how this applies to politics, war, climate denial, government policy and personal choices, before turning to the awkward fact that apparent irrationality can sometimes be strategically useful. Ultimately, we conclude that irrationality is messy, difficult to diagnose and often inseparable from the values and beliefs that drive people in the first place.
Transcript
Hello and welcome to the Cognitive Engineering Podcast brought to you by Aleph Insights and produced by me, Fraser McGruer. I'm here with Nick Hare and Peter Coghill of Aleph Insights. On this podcast, we look at a wide range of topics from an analytical viewpoint.
And today we're discussing Hitler's Alpine Redoubt. Nick, tell all. Go for it.
Speaker B:Yeah, well, as you know, we've decided to pivot to become a history podcast because we like the rest is history so much. So we've decided to emulate that by just. By just talking about random historical topics. Very good, very good. And this one is the Alpen Festung.
Speaker A:Festung.
Speaker B:No, Alpen Festung, the Alpine Fortress. So apparently. So I only know about this because I've recently finished reading Berlin by Anthony Beaver and I highly recommend it.
that Himmler put together in: Speaker A:Mountains and stuff. Yeah.
Speaker B:And in fact, after the war, apparently Verwolf, which was the kind of Nazi resistance movement.
Speaker A:Okay.
Speaker B:Did actually operate for a bit there. And I think.
Speaker A:Yeah, I don't want to sort of sidetrack us.
Speaker B:No, we should. This is. Yeah, let's go for it.
Speaker A:But is this the place that. Because he had two places, one of which I've been to.
Speaker B:So paying when you went on that pilgrimage to the sites of Nazi.
Speaker A:Yes, exactly, yeah. Saved up for years. So one was the. The Eagle's Nest, wasn't it called? Or the Eagles. Look out.
Speaker B:The Eagles.
Speaker A:Eerie.
Speaker B:The Eagle's Eyrie or something.
Speaker A:Is that the one?
Speaker B:There's also a Wolf's Lair, which was in Poland. I think so, yeah, it is.
Speaker A:And I've been there. So it's the Wolf's Lair. Wolfstrange.
Speaker B:Definitely an animal's house of some kind.
Speaker A:Well, when I worked in Poland, it's literally 30 minutes away from where I lived in Poland, and I took the family there a couple of years ago for a lovely day out. And it was. I mean, a lovely day out. It's a beautiful forest.
Speaker B:Well, this is nowhere near that. This is the mountains of Bavaria.
Speaker A:Yeah. Come on, get on with it. Back to the mountains.
Speaker B:And I think Goebbels actually did spread rumours that they were going to retreat there, I think to try and mislead Allied forces to some extent, but also to project more strength and durability than in fact the regime had.
So this had quite significant consequences, it turns out, because had the Allies tried to advance to Berlin more aggressively, then they might well have been accepting waves of surrender from the Germans, who were desperate at that point.
By the time the Russians were closing in, they were kind of desperate to surrender to the Western Allies because the Russians, they were perhaps quite rightly scared of what the Russians would do to them. It's possible that, you know, the US might have got to Berlin first. The consequences for Eastern Europe might have been very different.
And the West, I think the, well, Roosevelt in particular kind of thought that the Nazis would do the sensible thing and do that, that they would abandon Berlin, go to Bavaria and fight on there. So the idea was, well, you know, they didn't want to just do a big thrust to Berlin, they wanted to do a broad advance across Germany.
And Stalin exploited this because he, he knew that Berlin would be the site of an epic battle and exploited that and tried to, you know, pretend that he wasn't too bothered about Berlin. And Roosevelt kind of fell for it.
And one way or the other, what it meant was the, you know, the Nazis had their epic last stand in Berlin with the Russians. The Russians got to Berlin and then, you know, could kind of set the terms of how that would end.
Speaker A:Are you saying that maybe FDR thought that the Nazis would think rationally and that maybe the Nazis weren't thinking rationally and they ended up with an unintended consequence?
Speaker B:You've rightly detected where we're going. Okay, so for the Nazis and the Soviets, Berlin had this symbolism where they kind of got that that would be the final last stand.
ding here. Hitler, I think by:He wanted there to be this kind of Wagnerian epic. Gotterdammerung. Berlin was this sort of mythic centre of the Reich.
You might have heard of the Nero Decree, which was like, well, if, if, if the Nazis are going to be gone, we just destroy all of Germany, they're not good enough for us, etc.
And, and Lenin was supposed to have said, there's no evidence that he said this, but I think it was believed that Lenin said, he who controls Berlin controls Germany and who controls Germany, controls Europe. So there's this kind of symbolic value that I think we underestimated in the West. We thought, look, the, the Germans are going to want to fight on.
They're not going to just want to die for no reason. So they will do the sensible. Didn't they did from our point of view, a silly thing.
And Stephen Ambrose said this must rank as one of the worst intelligence reports of all time. That is the report that the Germans were going to move to the Alpenfestung. But no one knew that.
In the March of:Even Churchill was afraid of these developments, that is, the developments that the Nazis would choose to effectively die in Berlin and Hitler would choose to die in Berlin instead of going, you know, going and surviving somewhere. So, anyway, in some sense, we didn't anticipate how irrational Hitler would be.
I think we expected he would do the sensible thing, fight on in Bavaria. And actually, this is quite a common pattern.
When we're looking at what foreign governments will do and trying to predict what foreign governments will do, we often just assume they will do something sensible. And so before the invasion of Ukraine, there were quite a lot of pundits who said, putin's threats is just a load of saber rattling.
Putin's going to threaten invading, but he'd be mad to do it. Russia is going to be so much worse off if they do that. He'll never do that. And of course, he did.
More recently, Trump's attack on Iran 4D chess, which I don't think any analyst could see a way that there would be an end game where the US kind of straightforwardly got whatever it wanted out of this. You know, it's one of those. Just one of those things. We think. Well, we. What? How can this possibly make sense?
Not, you know, on your own terms, how can this make sense? How can this give you something that you want? So I think basically, we want to talk about what irrational.
What it means to do things that are kind of irrational, that are against your interests and. But actually, perhaps more importantly, for our purposes, what do you do when you're facing an adversary who's like that?
Speaker A:Right.
Speaker B:How do you manage fighting against someone who. How do you deter someone or coerce them?
How do you make them surrender or concede to your demands when they aren't even going to do things that are in their own interest? So it turns out that's quite a big problem, I think. Yeah.
Speaker A:I mean, how can you predict irrationality, let's say. Yeah, it might be one way to put it as well.
And presumably foreign ministries, diplomats, security, you know, ministries of defence, this is what they spend a lot of time thinking about, I would have thought, and I would imagine that the models that they have, there must be an irrationality factor, I guess.
Speaker B:I think, I think personally in my experience, we just, we underestimate it or at least we don't see it coming. I think there's a tendency to still think Putin's saber rattling over Ukraine.
It's about the domestic audience, it's about, you know, looking strong, it's about deterring the West. There's some reason for it.
I think there's a lot of analysis of Donald Trump which tries to impute a bunch of reasons for his behavior which may just not exist.
Speaker C:It's a common intelligence failure, isn't it? It goes hand in hand with assuming that the enemy knows more than they, they know.
Speaker B:Yeah.
Speaker A:Okay, so how are we setting this up? Where do we go from that? We've sort of framed the issue and how are we going to go about having a. So worrying away at this?
Speaker B:How are we going to fix the problem of irrationality?
Speaker A:How are we going to fix the problem of irrationality?
Speaker B:Easy.
Speaker A:Right?
Speaker B:How long have we got? We've got about another 20, 30 minutes.
Speaker A:Yeah, you got about 20 minutes or so.
Speaker B:Okay. No problems.
Speaker C:Well, what do we do for the last 10 minutes?
Speaker B:I propose on the basis of the Anna Karenina principle, which is that, you know, there's all kinds of ways that decisions can go wrong, is let's start by just saying, well what does it mean when you behave rationally? And then we can start looking at the failure modes and think about how we might fix them. Okay, sounds sensible.
Speaker A:Sounds very sensible.
Speaker B:Cool. Okay, so to make a rational decision, right? And this is just very bog standard decision theory.
If you want to make a rational decision, you've got to start with our old friend the utility function.
Speaker A:Oh, brilliant.
Speaker B:Right. That basically tells you of all the possible states of the world, how much you like them.
All in order says this is my absolute favourite one and this one I would hate.
Speaker A:It's about what you want, right?
Speaker B:This is just what you want. It doesn't care what you can have or anything, it just says how much you want things. Because we're going to worry about how we get them later.
But right now we need to have some preference over the states of the world. Then you have some set of beliefs about what the state of the world is like now. So you've got some sense of what is true now.
And now that's, you know, that's not necessarily straightforward to get that right. And a lot of the mistakes that we will talk about come from people under you Know, misestimating where they are now and then.
But then finally you've got to have a belief about your actions, about how your actions affect the state of the world. And once you've got those three things, you can in theory now just make the optimal decision.
You just look at all your actions and you find out what state of the world they give you and you go, well, I'll pick the top one.
Speaker C:Simp.
Speaker B:Easy rationality in a nutshell. Yeah, okay, brilliant.
Speaker A:Makes sense.
Speaker B:So all you have to do that and you will be sorted, all your problems will disappear.
Speaker A:Yeah, this is going to come in handy for me because I sometimes make mistakes about stuff, make bad decisions.
Speaker B:So we can talk about when you make bad decisions, which thing might be going wrong. Yeah, yeah, yeah, yeah. So I've actually got five common failure modes.
Speaker A:Okay, cool.
Speaker B:And we can use that then maybe to think about, well, how do we. How do we diagnose that? How do we see it coming? How do we make sure we don't fall prey to that? But also, you know, how do we.
How do we deal with someone else who's behaving like that? Got it. The first one is actually not even quite irrationality or a failure mode. It's just something we need to introduce, which is weird.
Utility functions.
Speaker A:Okay, go on.
Speaker B:Well, for one thing, Hitler, for example, he wanted there to be a thousand year Reich, right. Which was governed by Nazi doctrine.
Speaker A:So he knew what he wanted.
Speaker B:He knew what he wanted. And, but that, I think that's a bit of a weird thing to want. I mean, I don't. It doesn't sound like that much fun to me, to be honest. And there's.
But there's all like, I suppose less significantly, there are people who want to like do a deca man ultra triathlon. I mean, like that. Again, I wouldn't want to do that. But someone who dedicates their life to decide doing that might still be behaving rationally.
Right. It might just be. It's in the service of something which I think is weird.
Someone who wants to make a massive matchstick model of Chartres Cathedral or spend their life proving Fermat's last theorem.
Speaker C:Or Hume trying to scratch his finger.
Speaker B:Well, exactly. I mean, that's the thing. It's like actually you can kind of want anything you want.
Speaker A:Yeah, I want some wings. Right. That fix to me and.
Speaker B:Yeah, right. And that might be your big overriding dreams.
You're willing to make much bigger sacrifices for those things than, you know, we you or I'm Peter Or I might think was a sensible degree of effort.
Speaker C:To put even more tactically in the Hitler case we're thinking about is that if he retreats to the mountains, that's basically giving up. They're saying we are giving up on this dream.
Whereas if he stays in Berlin, even, even if there's a remote chance, there's still more of a chance that there might be some radical turnaround.
Speaker B:Yeah. So it's exactly. However small it is. Surviving, surviving in Bavaria, but without your Thousand Year Reich, it's no life at all if you're here though.
Speaker C:Yeah.
Speaker B:So survival is not actually as important as some chance of rescuing the Thousand Year Reich. Yeah.
I mean, there's people like, you know, Thich Quang Duc, who, who set himself on fire to protest about the persecution of Buddhists by the Diem government in Vietnam.
You know, I mean, we all want Buddhists not to be oppressed, but most of us wouldn't consider that that's a very high price to pay to raise awareness of something or, you know, Japanese troops in World War II refusing to surrender because honor is more important to them than personal survival.
Speaker A:Sure.
Speaker B:So just really, just to say, look, the thing is that, yeah, sometimes people want really weird things much more. You or I might want them.
Speaker A:Yeah.
Speaker B:Okay, so that's like weird utility functions, that's sort of thing one then. But the next one, which I think is probably just as common, I suspect might be more common, is actually just wanting the wrong thing.
Speaker A:But they're not the same then.
Speaker B:They're different.
Speaker A:Go on.
Speaker B:Well, wanting the wrong thing is thinking that some state of the world will give you utility. But it doesn't. Okay, classic example might be pursuing fame and wealth.
Because you might for some reason think, once I've got that fame and wealth nailed, I'll be happy. Well, you won't. I've got some bad news. You won't, Fraser, because wherever you go, there you are very wise, I think.
You know, in general, people overestimate how good, good things are and they overestimate how bad, bad things are. They overestimate things that they have now that people are loss averse, you know, and people forget about hedonic adaptation.
You know, it turns out that when you have your massive telly, pretty soon you. It's actually you don't really feel much better off than when you had a small telly.
Speaker C:You get used to it.
Speaker B:You get used to it, but also you'll be fine.
Speaker A:But I know you like to talk in personal terms. I prefer to do this in Strategic geographical stuff. Right.
But like, it's like wanting to, you know, land in the Dardanelles so that you can then capture Constantinople, which means that's how you're going to end the First World War.
Speaker B:Right. That's not necessarily the same thing. Is that not the thing, let's say.
Well, well, no, because wanting to end the First World War is the thing that they want.
Speaker A:That's the overriding thing.
Speaker B:Yeah. And actually that is in fact what you know, we wanted. Yeah, yeah, yeah. It's not like we were deluded.
We didn't think, oh well, when we win the First World War, you know, that'll be great. And it turned out we won the First World War and it wasn't. It was rubbish.
Speaker A:Right.
Speaker B:That's not what this is. That would be. We're coming onto this. It would be more along the lines of thinking an action will lead to an outcome and it doesn't.
This is basically being wrong about your utility function.
So, you know, it's, it's the, the issue about, about as I said, like thinking that you want something to happen, but when it does happen, it turns out it wasn't as good as you thought. Yep. Then we have. Being wrong about.
Speaker C:I find. I find a little. I find it a little difficult to differentiate between the two.
Speaker B:Yeah.
Speaker C:Be able to spot the difference even retrospectively between having a weird utility function. I wanting the right thing the wrong way, but. And wanting the wrong thing.
I think you might, you might sort of rationalize it post hoc quite easily or you might, you might find just that it's just a very difficult one. Pick how people's state of mind was in any given decision. So. Well, I'm not, I'm not saying that they're necessarily.
They're bad distinctions, but I'm saying it's. They might be moot because. Is very difficult to apply.
Speaker B:Well, there is a big difference which I guess we'll come on to when we talk about how you manage it, which is I might be persuadable that this thing I'm trying to do won't actually make me happy. And I think that's perhaps part of getting old and becoming wiser is starting to spot those kinds of things.
But there are some things which, no, I actually genuinely do want and they would make me happy. And I think there is a difference there in terms of how you might respond to.
Speaker C:I suppose, if you can. I suppose what it does help is inform the pre decision analysis so that you can be more honest with yourself.
Speaker B:Yeah.
Speaker C:And you can Examine your utility function knowing that these are two potential failure modes that are distinct.
Speaker B:Right? Yeah. And trying to separate them out.
Speaker C:Yeah.
Speaker A:So that just out of interest, is this your taxonomy?
Speaker B:Kind of, yeah. I mean it's still all based on standard decision theory, but I think I tried to sort of boil it down. Five main failure modes.
You could spin them out by, you know, having, well, your beliefs are correct but your beliefs about your beliefs are wrong and that kind of thing. But I think these more or less capture, I think most things that go wrong. It's one of these five things.
So the thing number three is being wrong about how things are. So that is you have just misappreciated the situation. You're deluded. I'm pretty. Hitler probably had a bit of this going on.
Probably thought the Wehrmacht was in much better shape than it was, you know. But also if you.
Hallucinations, propaganda, false memories, conspiracy theories might all might all make you think that the state of the world is different to how it is. And I guess, you know, a lot of when you have something like a war, well, both sides can't be right, that they're going to win. Right.
So I mean someone must be misunderstanding their strength more or less. So there's a lot of well studied cognitive biases about simply being wrong about things.
But I mean here again, it's not necessarily the case that you're being irrational all because you can be justified. You might be justified informing that your incorrect belief.
But anyway, so what I'm saying is you could be wrong but for the right reasons kind of thing. So that's being wrong about the state of the world. Then we have number four, which is being wrong about what works.
So this is just you getting your tactics wrong. So this is incorrect causal beliefs. You think a particular course of action will get you a particular thing and it doesn't, it doesn't work.
But so that's sort of.
If you've got bad models of the system or you've oversimplified it, or you've missed out some key variables or you know, you're kind of doing magical rituals, you've doing failures of imagination, you fail to think of the thing that would work. So you're kind of fighting the last war. I think this is a very common cause of bad policy, is just pursuing courses of action that don't work.
Speaker C:Often repeated as well.
Yeah, you see, you see the same bad policies to do with things like sort of organizational level, like management theory, bad practices getting repeated over and over, even though there's literature saying it's not a good idea doesn't work that way. Like yeah, performance reviews and things. So they're not. The evidence for them actually being any good is weak.
But it's still something that management people think they need to do or should do.
Speaker B:It is almost magical rituals.
Speaker C:Yeah.
Speaker B:It is like this is what they do. So we do this, this is what management people do.
Speaker C:And the other thing like political things like drug prohibition and certain sanctions and.
Speaker B:Communism because it just hasn't been tried,.
Speaker C:Hasn't been done properly. Yeah, yeah. I mean it's going to work. They persist. They persist and they, and they, and often they're kind of.
They come from a sort of conservative oversimplification of how the world works. But the evidence is clear that they don't work. You know a good policy research will say this has been.
There's loads of natural experiments already on this, we don't need to try it.
Speaker B:Well, Western medicine for hundreds of years did things over and over again that didn't work. So yeah, I mean you see this a lot.
Speaker C:Classic fighting the last war.
Speaker B:Yeah. Okay. So yeah. And again the last category, good old fashioned just being dumb. Just, just it's too hard to work out what to do.
Speaker A:Yeah.
Speaker B:So you vibe it, you YOLO it.
Speaker C:Yeah, yeah.
Speaker B:And, and lo and behold it turns out you haven't made the optimal decision. So. So there you go. What do you reckon? What do you reckon to my five part taxonomy of irrationality?
Speaker A:I like it, I like it.
I mean just that last bit, it just reminds me of talking to some climate change deniers and what I realized quite quickly is they couldn't bother be bothered.
Speaker B:Yeah, well let's diagnose that so that. What are they? They're what are they? What are they? If you like, what kind of action would they recommend?
What would be the action that they would recommend where we could say, well I can see where this has gone.
Speaker A:Wrong that they would recommend. So I don't understand.
Speaker B:Well, well it's an incorrect belief is one thing, but presumably that belief informs an action somewhere. You know, presumably they have a policy in mind or they think the government should stop doing something.
Speaker A:Yeah, they just think kind of inaction actually. Or like no, this is silly, it needs to stop all this stuff.
Speaker B:Yeah.
Speaker A:Okay.
Speaker B:Okay. So I mean I would diagnose that as being wrong about the state of the world.
Speaker A:Okay.
Speaker B:Or that, you know.
Speaker A:Well it is, but I think I do, you know what, I do like this taxonomy, but it's a little bit messier than and more interconnected than one might imagine, I would say. So, for example. Yeah. I think in this case, let's say climate change is a thing.
Speaker B:And they said. They deny that that's true. So they might say there is. There's no. The evidence isn't true. Right. The science.
Speaker A:Correct.
Speaker B:Yeah.
Speaker A:And so what you've. So you can say, well, they've got the state of things wrong.
But I would say that the reason why they've got the state of things wrong is they're not thinking it through very well, which I think is being dumb, let's say. So they're all kind of mixed up to bed together and a bit like.
Speaker B:I mean, being dumb will help you with all the other ones as well. You'll def. That will feed through. Yeah. Into being wrong about all kinds of other stuff.
Speaker A:But it's a bit like also divisions of subdivisions. Going back to this point when I was talking about the Dardanelles and you saw.
No, actually that this would go back to point one, but it's kind of a smaller version subdivision of, you know, because it was point one, which was weird utility function, I think. And I was talking about wanting the wrong thing, but.
But anyway, I mean, in this terms of what I would say they would need to do to fix this, which is they need to have the right tools. I could see that they just couldn't be bothered to think it through. And it was much more. It was more pleasing to the more reason.
Speaker C:Willful ignorance.
Speaker A:Yeah, it's kind of like, you know, it just. It didn't fit their world vision.
Speaker B:But anyway, computational constraints.
Speaker A:Computational constraints.
Speaker B:Kindly summarize it as that.
Speaker A:And then the other final thing I would say, and then I want to bring in Peter, is it reminds me, I mean, I don't know much about the law, but it's my understanding that the law.
I think in principle, the idea is that it tries to apply something precise and put in sort of, you know, definitions, let's say, and draw lines on what is a naturally messy world. And the law recognizes this, but this is what it tries to do. Okay. And that feels like this in reverse because you.
You might go, right, how can we make some good decisions? And let's, you know, let's. Let's write some stuff down.
Speaker B:Are we suggesting we might turn this into a law where it would become a criminal offense to be dumb?
Speaker A:Well, I was.
Speaker B:I would be. I would be potentially quite a fan of that. Yeah.
Speaker A:I mean, I'm not sure if I.
Speaker B:Would,.
Speaker A:But yeah, it's like, so it's kind of almost the reverse. And so you're trying to map things out but recognizing that, hey, it is going to get messy because it's dealing with stuff like why people don't.
Why do people get all this stuff wrong and do these silly things, let's say. So we've set out rational decisions, but irrational decisions. And you've come, you put in place this taxonomy of rationality, irrationality rather.
Where do we go from here? What's next? Are you coming in, Peter?
Speaker C:Yeah, I mean, I do like the framework. I think it. Having a framework before the decision is probably more useful than after the decision.
I find it very difficult to necessarily put historical examples into any one of these buckets, I think, but it's useful as an informative framework for analyzing those. You can say, well, it's probably a bit of this, a bit of this, a bit of this.
I find it difficult to distinguish retrospectively between, as I said, between 1 and 2, but also between 5 and 4 and 3. So how do you know what, you know, was the decision? Was it, you know, Dom is doing quite a lot of work here, right? So was the decision.
Was it, Was it what. What went wrong? Was it a lack of cognitive capacity? Was it, Were they over. Was a cognitive capacity overloaded?
Was it, Were they subject to cognitive failure such as various biases, you know, but it's also like, was it deliberately lazy? Like in your example of the kind of the climate deniers, maybe 5 needs splitting out as different kind of subcategories.
And 5 isn't really a thing that is on its own. It's more like a thing that would occur across all of the other ones, a causal factor for the other ones.
So, yeah, I like it as a framework, but I think it needs, it could do with tidying up.
Speaker A:Yeah, I think you did well, Nick. And you, you know, you've started it, you know, you started the conversation, got the ball rolling.
Speaker B: draft of the Rationality Act:Well, this is exactly because I have this feeling that if you. A lot, in fact, a lot of debates are about applying the wrong tool to the. To or at least trying to fix a problem which isn't the real problem.
By which I mean, you know, we.
I think I, I see a lot of people trying to convince someone of maybe a factual error where in fact, you know, the issue is, well, that person has a You know, actually just has a weird utility function. So you're not going to be able to persuade them out of wanting something by telling them other stuff about it. Do you know what I mean?
Like, let's say, you know, someone who really loves exercise and they love running and they just really enjoy it and you say to them, well, you're going to get arthritis if you keep running so much. Well, the thing is that they might know that they might not care. And similar to, you know, advice about giving up smoking.
Oh, do you know it's bad for you? Yeah, that is not why I do it. I'm not operating under a delusion that it's good for me, but you know what I mean?
Like, like a lot of, a lot of the way that people approach think seeing someone who is behaving irrationally and a lot of how we try to fix that is often just attacking the wrong bit of this.
Speaker C:Yeah, it's like kind of, it is partly who decides what's right. Inverted commas, what's the best thing you, you can't help when you're judging somebody else's choices. You are beholden to your own values.
Your values differ.
It might be that the chainsmoker just really does value the short term and doesn't want to get old and so has already made a internally rational decision that they will, they will die early.
Speaker B:Right.
Speaker A:But also I was think, what I was thinking about is when would we, as the voice of reason, want to intervene? Okay. And just thinking about that, I'm just thinking about how we use this, right?
Because this model feels pretty handy for governments, feels pretty handy for corporations and like every time we want them to use this.
But also I was thinking about, you know, democracy and you know, when you vote in a leader who, God forbid, might do these irrational decisions and how do we rationally, how do we make it so that we, the electorate, make a rational decision about getting someone rational in power?
Speaker B:Well, I find it very frustrating. I mean, it feels to me that when you get political discussions, when you get political debates, people munge together all of this stuff. They munch.
It's a mixture of like things that we want on one hand and things that we think will work on the other hand. And they're often just all squished up together.
And, and, and it's, and it's, you know, so for example, someone who really wants, for example, there to be much more equality is probably going to support some kind of forcible redistribution, higher taxes and so on. Well, they are two different Things you can want.
You can want there to be more equality, but also potentially want there to be, you know, to do that somehow through a liberal market, you know, that you don't. And yet. But it's always the same sort of. You get these. And you'll get discussions that.
That constantly dance around this question, well, are we agreed on what we're trying to achieve but disagree on how to get there, or do we. Or do we actually disagree on what we're trying to achieve in the first place? And I said just to see a lot of this and it's frustrating to watch.
Speaker A:I mean, a good friend of mine who actually listens to this podcast, and he's American, but he had a proper fallout, maybe not friends again with a fellow American.
Speaker B:Oh, yeah.
Speaker A:And I would say they're both on the. They're both on the liberal sort of spectrum of things.
And what they fell out over was the recent incursion or bombing, whatever you want to call it, of Iran by the United States and essentially the.
Not my mate, the other guy was angry with my mate for not supporting the strike or strikes and the reason being that Iran is a horrible dictatorship or blah, blah, blah, all that kind of stuff. And so you call yourself a liberal, then you should be supporting this because it's trying to take down a horrible regime. Regime.
And my mate, who I think would sit here quite nicely with us, was saying, yeah, it is a horrible regime, but just because you don't like something doesn't mean you have the means to. You know, there's many other reasons why you might not go ahead and do this, but just doesn't mean you can do it. I mean, physically is. You're not.
It might not be possible.
Speaker B:Yeah, I think that's a very good example. I see that a lot. You know, it's like, oh, you don't support this policy. You must secretly support terrorists.
Whereas it may just be a disagreement about the efficacy of the policy. Yeah. You know, and in that case, it's like, well, what? No one. No one likes the Iranian regime, but is this going to actually help?
Is it going to work or is it going to make them stronger? I mean, you know, you might. You might hate the Iranian regime, but still think it's not going to help if we bomb them.
Speaker A:So, yeah, so we're coming close to needing to wrap up, but we're talking about how, you know, how do you use this stuff or how do you react when you come across this irrationality? Peter, I think you've got a couple of Ideas around a couple of these, is that right?
Speaker C:Yeah, well, I think, I mean broadly the first two, wanting the wrong thing and having a weird utility function are probably the harder two to. To solve.
Speaker A:They can be quite fundamental, those.
Speaker C:Yeah. Fundamentally.
Hell, the reason I say that is because being wrong about the world, being wrong about how the world works, three and four and being dumb, they are largely kind of information availability. Computation availability.
Speaker B:Yeah. Kind of persuasion and education basically.
Speaker C:Yeah.
So 3 and 4 could be solved by just having more information and 5 can be solved by having more analysts, having better analysts or thinking more about what you're doing. So they kind of, they fit.
Speaker B:I mean it's still not necessarily possible to do. I mean you think about someone who's indoctrinated into a cult or something, you know, they just, they.
They're immune to you persuading them it's just not going to work. And I guess, you know, so, so there is to something.
I would imagine your climate change deniers are in a similar boat, you know that it's just there isn't some things you can say that will persuade them.
Speaker C:Yeah.
Speaker B:So it doesn't mean it can be done, but I know what you mean. Like it feels like. Well, it's still, there's, there is a common battleground which is. We can talk about evidence, but with the first two.
Yeah, it's a bit of a different story.
Speaker C:This question of how to want the right thing is harder than how to find out about how the world works. And yeah, a useful framework is, is Harry Frankfurt's second order desires framework, which is where the first order desires.
You see a cake, you say I want to eat that cake. But you have a sort of second order desire is where you. I want not to eat that cake.
So it's understood, knowing that you have this in the moment when you're making the decision, know that there are these sort of second order desires that can be violated, which is, I think you can think of it as like past you, future you. What would future you want to do? You would. Future you would rather you hadn't eaten that cake. So that's kind of your second order desire.
And a way, a sort of mechanism for doing that is perhaps thinking at the moment. It's like, well, what does, what would. Tomorrow's me say to me about this decision? Would he actually support it or not?
Speaker A:Say who you. How have you traveled in time? What. What's going on? Am I going mad? There's voices in my head. Sorry, go on.
Speaker C:Mad. That's all about Sort of a meta level understanding of what you as your actual desires are.
It's not like not just listening to the, the, the, the, the. Your brain stem desires about eating the cake. It's like what, what, what is my higher function, the ultimate desire?
And you can, on a personal level, you can sort of, you can, you can help yourself with this sort of thing by being explicit about what your values actually are and maintaining a sort of a crib sheet of what you, what you value in the world and then basing decisions, informal basic decisions, but informing decisions based on those higher level, non incidental goals.
Speaker A:Yeah, this feels like happiness versus pleasure.
Speaker C:But.
Speaker A:Yeah, sorry, go on. Yeah, yeah, you're quite right.
Speaker C:Yeah, yeah.
And the another thing is I think a lot of, a lot of these sort of wanting the wrong things are caused by what you could call the adaptive preference trap.
Where I think Nick alluded to it earlier when he's talking about policies where the debate about which policy is correct is informed by which policy is feasible. Quite to quite a great degree. So that you can think of the. I think it's Aesop's fable, the sour grapes. Yeah, the fox desert.
The fox sees some grapes up in the tree and he sort of initially goes, oh, they look like good grapes, but then decides that he doesn't want them and they're probably sour because he can't have them. Whereas actually maybe that's a failure of imagination about imagining how that policy could come in to be.
So rather than satisficing for a weaker policy, maybe actually pursuing the stronger policies,.
Speaker B:You get off his arse and build a ladder or something.
Speaker C:Yeah, yeah. Or go, you know, sort of team up with a bird or something. I don't know, something that happens in Aesop's favor.
Speaker A:This is what the world needs. The world needs to know about this stuff.
Speaker B:We should tell Keir Starmer.
Speaker A:Yeah, Fox Bird based solutions. Yeah, yeah, yes.
Speaker C:I mean what that looks like is when somebody appears to actually want something that could be a bad outcome, is they in reflection? They probably wouldn't.
Speaker A:A couple of things. But also there's metrics as well, which if you've got the right thing that you want to do and then you might be able to measure that.
Yeah, we did that or we didn't.
Speaker C:Right.
Speaker A:But there might be vested in. You might not want to know whether something worked out or not or how do you know it went south? How do you know it went wrong?
Speaker B:Well, this is why I think there's a lot of deliberate vagueness in government policy that we'll say things like. Ministers will say things like, we want stronger foundations for a better Britain.
And if you say precisely what unemployment rate are you going to be happy with, then they hate being pinned down like that because at some point they can be called to account and someone, someone can say, well, that policy you did didn't work. Right. And it's right there in black and white. It's much easier to say, well, we. We're all about prosperity directions.
And, and, you know, how do we know if this policy worked? Well, we don't really, because we didn't say what it is we're trying to achieve. So it's much easier to do that.
It's much easier to live your life being vague about what it is you want so that you can't say, oh, this, this thing I did didn't work.
Speaker A:Yeah.
Which is kind of weird because I just feel a lot of education we have in our lives, you know, until 21 or whatever, is about, is exactly the opposite of that, which is clarity and analysis and so on.
Speaker C:Okay, so very briefly, though.
Speaker A:Yeah, go on.
Speaker C:I think it's worth talking about when irrationality is actually beneficial.
Speaker B:Yeah, this is the real problem. Yeah, yeah.
Speaker C:And we've, yeah, we've alluded to it, like being. Appearing to be vague is kind of a form of this.
But a good example of this is the sort of mutually assured destruction situation and having a, Having a credible deterrent.
Speaker A:Sounds a bit mad to me, though.
Speaker C:Yeah. So it's like being, Being known. Like if you're in a.
If you're locked in conflict, being known to be a bit random, a bit irrational, like being out, you know, not responding in ways that your enemy expects you to can be quite beneficial because they, they don't. It gives them. No, it gives them less of a basis for planning against you. Another example is during negotiations.
Stubbornness and not acting in your own interest can actually be beneficial because it gives you more freedom in the negotiation.
You sort of get hung up on minor points that you don't actually care about too much and then use those as bargaining chips later on when you get to the stuff you do care about.
Speaker B:Yeah. I mean, literally. There's a famous game theory example, I think, from evolutionary game theory. I think it's called stag and dove or something.
But you can be. The problem is that if you're rational, you might back down when someone challenges you over your territory or something.
It might be rational to do that because you avoid a fight. But then, of course, all someone has to do is step to you and you run off and you're worse off because that person's just grabbed your territory.
It's better to irrationally defend your territory, even if you've got a chance of being killed, because then you're less likely to have to do that because people are deterred by it.
So this idea of having irrational desires like Hitler, irrationally wanting to defend the Third Reich rather than just going into hiding and living a nice life pretending to be a farmer or something, you know, he's got all this pride and loyalty.
You think of those Japanese soldiers who'd rather die than surrender and, you know, but actually what a big deterrent that was to us planners and things. You know, knowing that these. They just had a. Put a vastly bigger store on something that we would consider to be a value you would sacrifice.
You'd go, well, yes, I do kind of care about my honor, but I'm willing to swallow that a bit because I'd rather just surrender and have some hot soup.
Speaker A:Yeah.
Speaker B:And they didn't. Yeah, they didn't think that, you know, and so. Yeah, so. And so we have this problem where being irrational sometimes works.
Speaker A:Yeah.
Speaker B:And if that's true, then it's. It suggests that the optimal way to be is a bit irrational.
Speaker C:Yeah. That said, though, these. All these examples seem to me to be forms of performative irrationality rather than genuine irrationality.
Speaker B:Yeah. But then the problem is, Peter, if you're reality distinguish.
Well, if your irrationality is only performative and people know that, then they will exploit it. They'll know that you're only pretending to be.
Speaker C:Yes.
Speaker B:You've got to be only works if you actually are irrational.
Speaker C:You got to be consistent.
Speaker B:Well, then the only way to do that is to actually be irrational. That's the problem. Which is why I think we've evolved all of these, you know, which is why we end up with.
With the invasion of Ukraine and the Second World War and Donald Trump's attack on Iran. I mean, it's like. Well, actually, unfortunately, it sometimes pays to be irrational. I will, however, say I don't think it pays off in the long run.
I think the examples of when it goes wrong vastly outnumber the examples of when it goes right.
Speaker A:But also about the consequences and who pays those consequences. Consequences. All right. I think that's one of the things as well. Do you know what?
Speaker B:I think if, like, if Hitler had been sensible, if he'd said, look, you know, actually things are going a bit wrong, I'll settle for that.
Speaker C:Yeah.
Speaker B:Or at Least just sort of said, well, I tell you what, let's not invade the Soviet Union. Let's just be happy with what we've got.
We'll make a go of it and, you know, we're gonna live out your days as a nice sort of relatively successful big country. But no, he just had to go too far.
Speaker A:Would. You know what so much of. I've quite enjoyed this, actually, this last half hour or so, because a lot of the time we go at a subject and we.
Let's face it, nine times out of ten, we fix it. We wrap it all up. Right. Yeah, you know, this one, not so much. It's quite messy and it goes into so many other things.
Speaker B:Well, the crooked timber of humanity.
Speaker A:There we go. And it. But so many. So much of what we've been talking about reminds me of so many other episodes that we've done.
So we definitely did one on Future Self and we've done many episodes.
But right at the end there, what you were talking about reminds me of the episode which we did about talking to animals, which is, hey, if only Hitler could have been a bit more, you know, sensible about this. Well, then that would not be, you know, the tenants of, you know, National Socialism would. It doesn't kind of fit that. And it's a bit like.
Which French philosopher. Was it Sartre or was it the other bloke, I don't know, who talks about.
Speaker B:The two French philosophers who talked about,.
Speaker A:You know, if you could talk to a tiger. Well, actually, you're not talking.
Speaker B:Yeah. If a line could speak, we could not understand. There we go.
Speaker A:Tiger's line. Germans, French, whatever. But it's that kind of thing that unfortunately is wrapped up in being a Nazi.
Speaker B:Yeah.
Speaker A:Is there going to be a bit.
Speaker B:I agree, I. I think it's. It's very. That's. And that can be a bit headstrong. Well, that comes through very strongly from the book. In fact, there's a passage near the end.
Speaker A:What's the book again?
Speaker B:Anthony Beaver, Berlin passage near the end, which I thought was really interesting, where he says that the.
Just that sadly that a lot of these Nazi commanders, you know, when even after they were captured, not even the senior ones didn't really have any remorse for what they did and that their criticisms of what had gone wrong were based on that they didn't work.
So they said we shouldn't have invaded the Soviet Union, not because it was the wrong thing to do, but because it turns out we couldn't actually won and the Soviets were stronger than we thought, you know, we shouldn't have. We shouldn't have attacked France when we did, you know, because it.
Because then that turned the world against us, not because it was the wrong thing to do, you know, we shouldn't have persecuted the Jews because it turned international opinion against us. Yeah.
Speaker C:It turned out to be a bigger, harder problem.
Speaker B:Yeah, exactly. And. And I. I don't know. I mean, as you say, it's like. Well, somehow being a Nazi and doing stupid things at the same time appear to go hand in hand.
Yeah, Yeah.
Speaker C:I mean, it is. It is prefaced pretty. Pretty potty ideology, really. So, yeah, it's.
Speaker B:Yeah, you heard it there.
Speaker A:Condemned by Peter Coghill. Nazism, potty. All right, we'll stop there.
You've been listening to the Cognitive Engineering Podcast brought to you by Aleph Insights and produced by me, Fraser McGruer. If you haven't already, please like and subscribe. We try to release an episode every week or two.
If there are any topics you'd like us to cover, please email us at podcast@lephinsights.com thanks, as always, for listening. Until next time, goodbye.
