Predicting Russia
Does being a subject matter expert make you good at predicting events? This week, we delve into economist Tyler Cowen's blog on International Relations scholars and their views on Russia and Ukraine, and discuss international relations, analysis, and forecasting. We consider what makes for good analysis and the importance of having a sound methodology, diversity of thought, and understanding our own biases.
A few things we mentioned in this podcast:
- Tyler Cowen: How did the IR community get Russia/Ukraine so wrong?
- Conversations with Tyler
- The Good Judgement Project
- Expert Political Judgment: How Good Is It? How Can We Know? By Philip E Tetlock
- No True Scotman fallacy
- Cognitive Engineering: Hindsight Bias
- Poll: Will Russia Invade Ukraine
For more information on Aleph Insights visit our website https://alephinsights.com or to get in touch about our podcast email podcast@alephinsights.com
Transcript
Hello and welcome to the Cognitive Engineering Podcast produced by me, Fraser McGruer for Aleph Insights. In this series of podcasts we take a look at interesting topics and discuss what we think they tell us about analysis and decision making. I'm here with Jordan Fermanis, Tom Spence and Nick Hare of Aleph Insights and this week we're discussing getting Russia wrong. Jordan, lead us in, tell us.
Speaker B:So this topic comes from a blog post by Tyler Cowen on his blog called Marginal Revolution and it's about looking back in the lead up to Russia's invasion of Ukraine and trying to see, so he sort of sets out a list of IR experts, international relations experts, and sort of sets out their predictions for whether or not Russia would invade Ukraine or not. And then basically comes to the conclusion that most of the IR experts incorrectly say that Russia will not invade Ukraine and then goes into a list of possible explanations and reasons for why this might be the case, why these IR experts got that wrong. And so I think we want to talk about the idea of prediction in geopolitics but also look at perhaps his, Tyler Cowen's specific analysis and the way that he tried to approach this problem. Because I think we might have issues with his analysis, right?
Speaker A:Yes. And how he frames this. Well let's go straight into it. Tom, what do you reckon is the article?
Speaker C:So I have a few issues with the article. I'm not some kind of defender of international relations but I think the premise of the article is flawed.
Speaker A:Who's the bloke again? What's his name?
Speaker C:His name is Tyler Cowen. He's an economist. Okay, right.
Speaker A:I think that reacts to the norm. Excuse me. But I remember actually because I got an MA in international relations, I remember an economist slagging off international relations saying it was... Was it me? Probably me. No. It was like most economists talking a load of rubbish and just saying that, you know, IR was viewed in the economics academic community as a soft option, as a soft subject.
Speaker C:Don't worry, Fraser. I come from a natural sciences background and I can tell you we say the exact same thing about economists.
Speaker A:Yeah, about any social science. Yeah, there you go. So yes. Okay, so he's really an economist. Does he tend to talk about international relations or does he talk about all sorts? Like a lot
Speaker D:of economists think he's qualified... He's not talking about international relations. He's not. Sorry. He's talking about international relations forecasters. There's a big difference. He's not weighing in on international relations. He's saying here's some experts, inverted commas, and here's what they said. And lo and behold, a load of them are wrong. I'm pleased you said that, actually, because it's been ages since you and I've had a little
Speaker A:tiff. I feel a little bit, oh, that feels a bit better.
Speaker D:No, but it's just that this reminds me...
Speaker A:You're picking whatsits, you know, you're splitting hairs.
Speaker D:No, this is like if someone says, you know, I'm an expert at fixing ovens. Yeah. And you look at the last 10 ovens they've tried to fix and they haven't fixed any of them. Yeah. And I say, I don't think you're terribly good at fixing ovens. And how would you feel then if he said, well, what do you know? You don't know anything about ovens. You say, well, I don't need to. I can just look at your record with ovens and it's poor. Right? It's perfectly good to have an opinion on how good a forecaster is, even if you know nothing about the subject matter. I've forgotten what...
Speaker A:I'm just saying this, laying that down. I'm just going to agree with you because I forgot what the initial thing was. Well, because I'm right. Well, mainly just to Tom.
Speaker C:Yeah, over to Tom. So I do actually agree with Nick. I think you can judge how... You always do. You can judge how good a forecaster someone is without necessarily having subject matter expertise. However, I don't think that that is what Tyler Cohen has done in this article. Yeah. So he has essentially cherry picked a few examples. He's actually got two examples here of IR professionals, IR experts who are predicting things wrong. He then has a few examples of people who are not IR experts who he says predicted it right. Yeah. And then he says, well, I couldn't find any IR experts who predicted it right. So there must be none. I bet there are loads. Yeah, he didn't look very hard because you could look in the time frame he's talking about. Foreign Policy published a survey in which surveyed international relations experts in which more than half of them said Russia was going to invade Ukraine within a year. If you want to look at further scales, you can go back to...
Speaker D:Well, that's saying they're as good as a coin flip.
Speaker C:Well, so that's not including the unsures. The people who said they wouldn't invade was only 20 percent. So it's actually it's better than a coin flip on those who...
Speaker D:More than half said they would and 20 percent said they wouldn't.
Speaker C: ack to literature since since: Speaker A:What does Tetlockian mean? What does that mean?
Speaker C:So Nick is probably best placed to explain Phil Tetlock.
Speaker D:Phil Tetlock's one of my personal heroes. And he is the guy who, well, he was the head of the Good Judgment Project, which was an attempt to measure, essentially asked thousands of people to put probabilities on things. So, you know, here's a scenario. How likely is it to occur? And then he looked to see what seemed to drive people's ability to put accurate numbers on those things. But actually, that was the following on from about 20 years of work that he's done, that he'd done, which he published in a book called Expert Political Judgment. Very much looking at this. But the key thing was he was giving people the scenarios. So I think this is probably important. I don't know if this is what Tom's going to touch on, but he was, they generated the scenarios and gave them to people and said, here's a scenario. How likely is it? It's a different situation when you're looking for things people have gone out of their way to tell you. So if someone's written an article because they think something is important.
Speaker C:Or conversely, that they think it's not important.
Speaker D:Right. But I mean, if somebody's written an article saying Russia won't invade Ukraine at the very least, they are looking at the issue of whether Russia will invade Ukraine. They're not looking at, you know, whether Japan is going to, you know, recommence large scale whaling. I mean, they're looking at the right thing, but they're forming the wrong opinion.
Speaker C:And that's why I think, for example, this survey that I looked at is a lot more scientific than this article, because everyone was asked the same question. But crucially, what Tetlock didn't do was go and find a few predictions, say, these are right, therefore all predictions are right. Or conversely, these are wrong, therefore all of them are wrong. Tetlock actually had methodology that was defensible. Yeah, yeah. I also, on a more general scale, one problem I have with the article is that international relations as a discipline isn't really in the business of making predictions. It doesn't claim to be. It's a bit like saying to an economist, why did you not get the price of oil right last Friday? That's, it's an analyst's job to forecast events. An IRR theoretician's job is to provide frameworks that explain why events happen and why causal relations exist.
Speaker A:Yeah. So the guys that he picked and said, oh, look, see, this guy didn't get it. Were they, so as you say, they're international relation theorists, let's say. But were they people who were predicting, saying there will not be a Russian invasion? Or was he just saying, hey, this guy didn't mention it was going to happen?
Speaker C:No. So he's picked a couple of people who specifically predicted, we think nothing's going to happen. He also goes to great lengths to talk about how smart they are, so that he can then say, look, the smart people got it wrong. There's also a little bit of a no true Scotsman fallacy in here with some of the people he mentions that did predict it. Then he says, ah, but of course the IRR community doesn't take them seriously. Okay. Again, with no real evidence backing it up. So I think what we should talk about here is how can you actually say if predictions were good, if a discipline or a group is good at predicting things, which very much comes down to Phil Tetlock's methodologies.
Speaker A:Okay, good. So you've both led us nicely in there with the article, and we've already debunked that article, fine.
Speaker D:Well, I don't, let's not be as, let's try not to be simplistic. I mean, look, there's What? There is, it is the case that, I mean, one of the things that Tetlock found was that expertise is uncorrelated actually with forecasting ability. Right. Now, I think one completely legitimate criticism of the Good Judgment Project is that it perhaps made a bit too much of forecasting. The reason that it focused on forecasting is because you measure it. You can ask someone for a number and then you can say this happened or it didn't happen. It's actually not even as straightforward as that because it's sometimes quite hard to find out if something happened. But just put that aside for one sec. It measured the most measurable thing about analysis, which is, you know, here's a scenario, what probability do you put on it? You get a high score if you had a high probability of something that happened, and a low score if you put a low probability on it. And then you work out who scored most highly. That's all great. There's a whole bunch of other things involved with being an expert that Tetlock didn't try to measure and has assiduously avoided pronouncing on, which is, do they add value in other ways? Tom touched on this question of kind of understanding. They have models of understanding the world, which, you know, might be helpful in some way. However, I think, and I think this is Tyler Cohen's point here, you know, actually, what's the point in having all those models if they don't help you be right more often about the world? I mean, so even though he's making the point clumsily, there is a point there to be made, which is, you know, if at the end of the day, you're not, you're no better than the man in the street at saying whether Russia is going to invade a neighbouring country. What is the point of all that expertise? Okay, I'm not saying there isn't one. What I'm saying is, what is it? Yeah, Jordan. And I think on predictions,
Speaker B:even the international relations experts might, it might not be sort of what their best place to do. And we might sort of assume that they don't hold much meaning. I think prediction and forecasting is important in the way states deal with each other, because they make decisions based on what they think other states are going to do. Which is why I think in intelligence analysis, it's important. And it's also, I guess, important to gain insight into potential future policies, and how they might play out in terms of scenario generation as well. And they can be useful for businesses as well, in predicting. So businesses might want to know how politically stable a country is before they invest in it, for example. So I think, I mean, it's probably another question whether or not international relations experts are the people that should be making the predictions. But I think prediction does sort of play a part in our world.
Speaker A:Okay. Well, look, what direction are we going? Do we want to drill into this? What is the difference between right and wrong? Let's just talk about Russia.
Speaker D:Okay. All right. So let's keep it on that. There's a bit, something to say about Russia. Well, okay. So if I was one, first of all, okay, I can tell you what a bad prediction is, is when someone says this will happen or it won't happen. That is always wrong, right? The probability of anything is never 100% or 0%. There is a correlation between people who do that and wrongness, right? So forthrightness, strength of opinion is correlated with being a worse forecaster. And at the very least, you can say, well, if, you know, it's almost like, you know, there's a reason if a cynical way of looking at it would be to say they're gambling, they're gambling on looking like a profit. You know, if I say this thing that you guys are all saying is happening, isn't going to happen, and it doesn't happen, bingo, I get to, I get to tell all my mates in the, you know, International Relations Club, that I was the guy who was right that time. And, and actually, you know, it's the, it's right one time in 100. That's all you need, because most people don't care about the 99 times you're wrong, they're not going to follow it up and hound you. But you know, occasionally you get a sort of, you know, Nassim, Nicholas Taleb, who will make an outrageous prediction, which will come surprisingly true, because they hit the right spot at the right time. And, you know, they'll become a celebrity from it. And that's perhaps cynically what these people are doing. But let's make their case for them. Perhaps they were right to be wrong about Russia. So that's one of the angles. Explain that a bit more. Well, maybe, maybe this was actually a genuinely really surprising and unlikely thing to happen. Yeah. And if I was trying to make that case, which I don't think stacks up for one thing, because Russia was already had already invaded Ukraine before, right? And they were already occupying parts of it. And it's not perhaps that much of a stretch, right? So there's lots of reasons to think that you, you shouldn't be terribly surprised by it. But I think one of the key, one of the things that Tetlock found was characteristic of sort of super forecasters, so the people who are particularly good at making forecasts repeatedly on a range of different things, was that they would look at, first of all, a kind of statistical approach, base rate approach of how often does this kind of thing happen in general? Like how often does one state invade another? Start there and then start asking questions about the specifics of the situation. But the other thing that they look at is interests, like the major powers. What is it in their interest to do? Because by and large, people act in accordance with their interests. And in this situation, I think it's hard to see what Russia is getting out of it or what they could even have hoped to get out of it. There's one very specific scenario I think where they gain, which is a very rapid collapse of the Ukrainian government and an easy steamroller of the army collapsing them, stick a puppet guy in there, hey presto, and before the rest of the world can react. That's the one way I think it could have worked out in their favor. But that's very unlikely to happen. And it's reasonable to think that the Russians ought to know that, that they ought to know this is gambling a lot on that outcome. Because if that isn't what happens, then pretty much all outcomes sort of look a bit like what they've got. And what are they gaining out of it? What cost? So a rational cost-benefit analysis ought to say that they aren't going to do it. They're kind of mad to do it.
Speaker A:Well, that's interesting, though, that you say that. And I remember, remind me the name of the Canadian bloke we had on here, the Canadian military academic type person.
Speaker D:Do you mean Dr. Kristian Gustafson? There you go, Dr. Christian.
Speaker A:I think he's officially British now. Yeah. Because I remember we talked about, I don't think we were, we were talking about Russia. And we talked about what does it mean to talk about Russia? We talk about them and it and him. And we talk about Russia as if it's a person, right? But one of, and I presume that the method that you're talking about, the good judgment, is how do you assess methods on the other side and decision making on the other side? Because that's something that definitely plays in to what happens here. Because Russia doesn't make decisions like China does, well, more closely or like America, or I think that's part of it. Tom.
Speaker C:So this is something that I thought was quite an interesting topic. And so I looked into it a bit more. If we do accept the premise in Tyler Cohen's article that IR got it wrong, and we want to look for explanations of why, he presents a few that I don't really buy into, but one that I think is more reasonable. And this is a blind spot that IR knows it has, is that in a lot of the West, IR is taught in a very specific way. You learn about realism, you learn about various different schools of IR. A normative approach. Exactly. That are Western schools of thought. What far fewer people are taught are different IR methods from around the world, and different IR schools from around the world. So Russia has a very rich tradition of IR scholarship, but it's primarily published in Russian language journals and publications. So a lot of it doesn't actually diffuse out into Western IR discussions. And so I think this comes a little bit to Nick's point in that when we're looking at the theory and what would be a rational decision, we're looking at it in the framework of our own theory. But traditionally, Russia has a different set of theories in international relations that their academics and their scholars who much like academics and scholars in the West have influence on government decisions in various ways, through think tanks, through official posts, et cetera. And that lack of understanding between the two, now to clarify, there is some understanding and it's something that has been identified, more people are trying to work on it, but it is still an issue. So for example, there are three main schools in Russian international relations, which are Westernism. So that's the sort of thing that Kozyrev was a big proponent of after the fall of the Soviet Union, which is a pro-West, sees Russia as being at its core Western. There's statism, which sees Russia as being at its core Russian and different to the other, which the West is the other, historically the East and the Tatar yoke that the Muscovites were under is the other. And Russia sees itself as independent of both of those. And as such needs various Russian characteristics to its international relations. So strong centralized state, et cetera. And then a third is a kind of civilizationist approach, which isn't really something you see that much in Western IR. And this is where you get the schools that are along the lines of Eurasianism, which is quite prominent in a lot of, a lot of high profile Russian thinkers at the moment, or in the past, Panslavism around the time of the First World War, for example. And these don't really have direct analogues in Western IR tradition. I think we're talking about two different things,
Speaker A:but anyway. Yeah. Well, because I think something that would be accounted for in Western school of thought or thoughts of thought of international relations would be to what degree, you know, decision-making is centralized or not, for example. And also, and one of the things that would have analyzed, for example, the more or less rapprochement between the US and China, a lot of that was down to personalities of leadership, right? And so that's a potentially a positive thing, but a negative thing is it's not in a structured framework. And so therefore, when those personalities subside, then you don't have the framework for the relationship to continue. And so that's kind of what's happening here, I would say. And that was the point I was trying to make, which was that how can you predict if you've got an unpredictable opponent, let's say, something along those lines, or how they make decisions is different. Have we gone down a wrong rabbit hole here? Do we want
Speaker C:to bring it on? I think that is the key. A lot of IR scholars in the West don't account for the fact that IR scholars in Russia think about things in different frameworks and make their decisions differently. And so I think you're right to raise that. I think that is a real issue.
Speaker D:Sorry, Jordan, I'm still formulating a thought.
Speaker B:I was just going to pick up from Tom's point, which I think is, I think the Tyler Cohen's article with his blog post sort of brings this to light is that a lot of the people that he references, the IRS experts are American, or they're from the West. And they're not necessarily subject matter experts in Russian politics or Russian affairs either. They're more generalist. And they sort of weigh into all sorts of different geopolitical debates. And I think that is, to just pick up on Tom's point, I think that is probably one way that his analysis is lacking is that it reverts to focusing on sort of so-called IR experts in the West and at the expense of perhaps people inside Russia that might have more intimate knowledge of these events.
Speaker D:I've got two things to say. Yeah, go on. First of all, IR be damned. The IC, the intelligence community, particularly in the US, nailed this one, right? And they're not, you know, they're not, I mean, I would imagine some of them are probably, you know, they're good old-fashioned criminologists and stuff. But, you know, I suppose what I'm saying is that Russia was planning to invade. It's not something you require a kind of detailed knowledge of the threads of Russian historical materialism to get, right? Putin is clearly an acquisitive, you know, imperialist. And he's invaded other countries before. And he likes having countries like Belarus at his beck and call. And he's said as much as, you know, Ukraine's a silly country, it's made up and it ought to be part of Russia. You know, it shouldn't, it's not that, it's not an obscure and complex and weird thing you have to look at. That's my first thing.
Speaker A:Do you want to interrupt on the interject? Just let Tom come in.
Speaker C:I've got a very quick response to it. Yeah. Which is that I think that is almost the same as my earlier point, which is that predicting this is an analyst's job, not a theoretician's job. And you're saying the analyst got it right.
Speaker D:I guess I feel like I know who I would call if I want to know what Russia is going to do. That's all. Agreed. The other thing is just, it's about this problem of hindsight bias, which I think is so prevalent. And it's prevalent, you know, after the event happens, there's a big temptation to look at it and go, it's kind of obvious that Russia was going to invade. How could we have been so silly? Of course they're going to invade. Like, here's all the reasons why they were going to. And likewise, actually, you know, we can sort of, there's a bit of hindsight bias going on here where we're going, well, you know, in as much as there was uncertainty, what did we get wrong? Or, you know, perhaps we should have paid more attention to Russian schools of thought about international relations. Let's say, look, let's run with, okay, we were kind of more surprised than we should have been. And we should pay more attention to Russian IR theories as a result. The flip side of that is, if Russia hadn't invaded, would we now be saying, you know what, we should pay less attention to Russian IR theories because we were right. And it turns out those Russian IR theories aren't terribly important because actually our way of analyzing IR has won this time. Well, no, we shouldn't be doing that either. So we, when things like this happened, it's still only, it doesn't represent a hugely informative outcome, I suppose is what I'm saying. And we shouldn't be tempted to throw out or usher in new theories as a result. We should say, oh, this gives a, perhaps this gives more weight to the idea that we should be paying attention to Russian IR theories a bit more. And I say this as someone whose knowledge of Russian IR is based entirely on what Tom said five minutes ago. Or, you know, perhaps a bit less or perhaps this adds a bit more weight to this theory or that theory. But what we shouldn't be doing is looking at this outcome and going, well, this shows that IR is a load of rubbish or that anything else is a load of rubbish or is obviously right.
Speaker A:But also, yeah. And also what makes me, this makes me think about is certain questions like, well, what do we want to know? Why do we want to know that? And how do we define whether that is right or wrong, the outcome, and even whether right or wrong are the correct terms? Look, we're sort of most of our way through this. Where do we want to go with this? What do we want to say? How do we move on from what we've said so far? Have we covered it? We've done it. We've nailed it.
Speaker C:Should we discuss how would we assess whether international relations got it right, given that we don't have the ability to specifically say, here's a question, predict it? Yeah. How would we assess this natural experiment? Would that work?
Speaker D:So Fraser, you didn't publish any articles, to my knowledge, in any reputable journals weighing in at all. Lots in Russia.
Speaker C:Why did podcast hosts get Russia wrong?
Speaker D:Yeah. Well, the thing is that, no, because he didn't get them wrong either because he was silent on the whole issue. Was he more right than an IR expert who came out and said this is almost certain not to happen? Or should he just not receive any marks? Should he receive marks for not having said something wrong? That's the first question. I mean, because that's really actually a hugely significant and difficult question in this domain, is how do you mark someone who said one right thing once or no right things once with someone who said 100 things, 80 of which were right?
Speaker C:If we go by Tyler Cohen's methodology, Fraser got it wrong because, as he said, he couldn't find any IR people who got it right because he wasn't looking very hard. Therefore, they all got it wrong. I couldn't find anything from Fraser getting it right. So Fraser was wrong.
Speaker A:But also, and I don't know if this is what you're talking about there or if we've talked about it before, but I remember thinking about chief economists at investment banks. And people always want to know, oh, is there going to be a recession? Right? And I remember there was, I forget the name of the economist or whatever, a major bank was known as the doom monger of whatever, because every year he would always go, oh, there's going to be a recession. I mean, he would always say that. Of course, there never was. But people didn't really remember that. What they remember was when there was a recession and he got it right. And everyone was like, we should have listened to him.
Speaker D: oreign office official in the: Speaker A:Yeah. So I've got a comment and a question and another comment. So my comment is actually I'm a super forecaster. Are you? Is that correct now? Yeah. So I think you may recall that I correctly predicted the election of the next US president.
Speaker D:That's right.
Speaker A:Right. The most recent, the current incumbent. Yeah.
Speaker D:No, I think, didn't you get, say, oh, was that Trump Biden? Yeah. All right. And you went for Biden because Chris had gone for Trump.
Speaker A:No, what happened is, I do listen on these podcasts. And you guys made some really good, I felt really solid analysis about why Biden would win. Okay. And I went brilliant. And so I put some money on it.
Speaker D:It was a bottle of bourbon, wasn't it?
Speaker A:No. Oh, that was something else, I think. Oh, okay. And I won myself quite a lot of, you know, reasonably amount of money. Ah, yes. Which I'd been saving up for a double base. Do you remember that?
Speaker D:And now you've got a quadruple base.
Speaker A:Yeah. I did the same. I did the same with this, sort of. Not in, not, you know, I don't want to make light of what's a sort of, you know, not a nice situation. Eurovision. Okay. It was really clear to me what was going to happen with Eurovision and who was going to win it. And the odds, I mean, Ukraine was way out in front in how it's going to win. And I just knew that, and also knowing how the voting more or less works for Eurovision, that the popular vote is really important. And so, yeah, I put a pretty substantial amount of money, actually, on Ukraine winning. The house. Not quite that much. That's interesting.
Speaker D:A frightening amount, I'd say, because it wasn't worth it because they were so clear favourites. Right. It's interesting because for exactly the same reason, I put not quite a lot, but about 20 quid on Ukraine to lose because I thought the popular wisdom is wrong more often than these odds are giving me, right? Because the odds were ridiculous. It was like 15 to 1 or something. It's like, look, I'm sorry, but there's more than a 7% chance that Ukraine won't win for some reason. You know, that maybe there'll be, you know, there'll be some freak. It turns out that, you know, maybe the juries are going to be swung by Ukraine, but actually the popular vote, maybe there's a lot more Russian sympathisers. That's the thing I was worried about. I was worried about the Russians messing with the system. Or not even that, but even like people who genuinely actually aren't just sympathy voting.
Speaker C:Maybe people take Eurovision a lot more seriously as a musical contest.
Speaker A:Yeah. And they love the song from Moldova this year or something. It's an amazing song.
Speaker D:So my feeling was that, well, even though Ukraine was probably likely to win, it wasn't that likely to win. So I bet against it. But yeah, I mean, the feeling that, well, actually, oh, I'm going to vote on musical merits, but all those other stupid people are going to just vote for Ukraine because it says so on Facebook. I had more faith in people than that, but that faith was sadly destroyed. And it turns out people really do just vote on that.
Speaker A: he economy of Venezuela circa: Speaker D:Yeah, but you were right in the long run in that the Venezuelan economy is hardly the powerhouse. Exactly.
Speaker A:In my opinion. And so I couldn't quite understand why it was growing, but well, I do. It was just such an unpredictable-
Speaker D:Yeah, the market was wrong and you were right.
Speaker A:But the other thing is I would always continue to be saying like at some point soon in the next year or two, the shit's going to hit the fan in Venezuela, both politically and economically. And it never quite did when I was an analyst. But then sure enough, within five years, I mean, it's horrible.
Speaker D:Sure enough, five years later, the Beatles split up. Yeah.
Speaker A:But yeah, it's a complete basket case in many ways now. And so I was definitely right. It's just a question of timing. Very unpredictable economy. Waffled on there for ages. Worst ever forecast that you've made. Doesn't have to be to do with countries and economics.
Speaker D: ng. Yeah, well, there was the: Speaker A:Okay.
Speaker D:Which is that people were- there was a lot of soul searching about is this the twilight of the market economy? And, you know, is this the end of capitalism, etc.? And I was much more upbeat than I probably should have been about the economy. I was saying, you know, this is like a financial phenomenon, which we're seeing. The idea that this is going to have- everyone thinks that the markets are the same as the economy, but they're not. One is about fundamental production and one is about pricing. Okay, we've got the pricing a bit wrong about the future, but it's not going to actually affect the economy. But that turned out to be incorrect. It did. You know, but plausibly, okay, it was also a symptom. Sorry, how do you mean you were incorrect? What do you mean? Well, I'm saying that there is- I sort of said, well, this isn't really going to affect the economy. Oh, okay. No, this is going to affect the markets. The journalists are really into markets, but it's not. It's a purely financial phenomenon. I didn't advise the Treasury of that or anything. I wasn't paid to do that. But the other thing about- you know, my point was, look, people believe in the capitalist economy economic model because it works. It's still better than any of these other models. But then I think we can say that there's been a sort of decade of populism, plausibly as a result of the sort of apparent failure of capitalism. But I would still stick to my guns, though, on that. It's still the best system we've got. You know, I don't think those new populist countries are doing any better. But anyway.
Speaker A:Yeah, okay. Good. Tom or Jordan? It can just be about what the weather is going to be.
Speaker B: mind is I remember saying in: Speaker A:Okay. And also we're Tottenham in the top four. Yeah. All right. No need to rub salt in the wounds. Let's move on from that one. Yeah. Okay, good. Well, also, I suspect there's some- because there's emotional stuff going on there. And especially with Arsenal, who are clearly a pretty awful team, but you can't help but the emotions get in the way of it. Tom?
Speaker C:Well, yeah, I think the same. I think where- I generally try to be in the business making good predictions, especially if Nick's asking, because he employs me to help make predictions. So yeah, I think most of the times where my predictions have been worse are where I have let sort of overconfidence or a bit of Dunning-Kruger in subject matter expertise scupper me. So whether that be an emotional attachment to a football team, I once put money on Mohamed Elneny to open the scoring for Arsenal. Terrible bet. Why would anyone bet that? He scores one goal a season, maximum. I've definitely tried to make some predictions about how things would unfold for Lebanon's economic and political crises. And generally been wrong with those because I feel as though I know more than some people. And so I feel overconfident. And that's a mistake.
Speaker D:Yeah. And that's the first step in the journey, isn't it? Realizing where your biases are.
Speaker C:And we've come all the way back round to Tetlock and expertise being uncorrelated.
Speaker A:I like the way that, you know, that what we've realized is that out of all of us, I'm the super forecaster. Because I've got a hundred percent. Did we realize that? Yeah, no, we have. We've all just agreed that. I've got a hundred percent record. Positive record.
Speaker C:Wait, didn't you ask what our worst forecasts are? And then talk about how you were really good at forecasting.
Speaker D:You forecast that the Venezuelan economy would collapse. You were only six or seven years early. Yeah. But no, never mind. Let's run with it. Two out of three. Yeah, not bad.
Speaker A:Yeah. OK. All right. Let's stop there. Thanks, as always, for listening to the Cognitive Engineering Podcast. I'm Fraser McGruer. We've been here with Nick Hare, Tom Spence and Jordan Fermanis of Aleph Insights. Until next time. Goodbye.
