Fragility

Speaker A: 00:00:00

Hello and welcome to the Cognitive Engineering Podcast produced by me, Fraser McGruer, for Aleph Insights. In this series of podcasts we take a look at interesting topics and discuss what we think they tell us about analysis and decision making. I'm here with Peter Coghill, Chris Wragg and Nick Hare of Aleph Insights and this week we're discussing how Cloudflare killed the internet. Peter, I've no idea what all this is about. Enlighten me.

Speaker B: 00:00:29

Cloudflare didn't kill the whole internet.

Speaker A: 00:00:30

Good.

Speaker B: 00:00:31

They had a major outage on their own platform. What is Cloudflare? So Cloudflare provide various hosting services and one of the big things that they do is website hosting. So they have what's called a distribution network. So if you host a website on their server, their distribution network will replicate that website around the world on other servers such that users of your website get fast response times from your website. Makes sense. And it was this particular service of theirs that got knocked out. So it's not the whole internet. Most of the internet was still working. It was just users of Cloudflare and any sort of third party users who were using things that had been built on Cloudflare that had problems. So basically what had happened was that they were simulating rolling out of an update. So they continually make updates every day, probably, as many providers do, to address security concerns or vulnerabilities that have been discovered in their code. So it's a continual development cycle that happens all the time. So they're pushing out an update and they're pushing it out in what's called a simulated mode, which is what they thought was a safe mode for trying things out. So basically what happens is it's designed and tested back in their headquarters, which is designed to iron out any potential problems and then they push it out in simulated mode, which means that real traffic gets routed through it as if it was for real, but it doesn't actually affect the result. So they can see how it would affect the system. But the problem was that this particular update had got an error in it, which caused, even in simulated mode, caused the CPUs on the servers to max out and consume all the system resources, which meant that they became less responsive to management traffic and all sorts of other things. So basically the whole thing just locked up. And this was a routine small change that caused this. And in a way that they hadn't really anticipated. I don't think they'd ever appreciated that a change like that could cause such a fundamental change, cause such a fundamental problem with the servers. It was only really supposed to be affecting the traffic going in and out of the servers. But they fixed it. They fixed it very quick. And what's really quite interesting and fascinating is that they gave a very detailed postmortem on the event, published it on their blog for everyone to see, which is quite unusual for big tech giants to do. I can't imagine Facebook or Google really doing that, really being that open and transparent about their own mess up and saying, sorry, it was our bad and this is exactly what happened. But yeah, the problem started at 13.42 Universal Time on the 2nd of July. They diagnosed it 20 minutes later at 14.02. By 14.09 they got a temporary fix in, which meant it all started working again. And then by 14.52 they'd got a proper fix in place. So incredibly quick, really, but still that was still like 27 minutes or so before when people were being affected by this problem.

Speaker A: 00:03:55

And what is our interest in this? Why do we want to talk about it?

Speaker B: 00:03:58

Well, I think our interest is that there seems to be a fragility to the modern world caused by these highly complicated systems that rely on each other to operate.

Speaker C: 00:04:14

And so, well, okay, so thinking about then, is it too fragile? Are we too dependent on it? Do we need to do something about it?

Speaker A: 00:04:21

Well, that being the case, even though it was a quick fix, what were the effects then? What happened because of this?

Speaker B: 00:04:30

Well, so the effect that most people would have spotted really is lots of websites not being available. And I imagine lots of web apps also went offline. So there'll be things on people's phones, on computers that rely on services that are published on Cloudflare server. So I don't think any banks are on there, but imagine online banking. If a part of your online banking system relied on the Cloudflare server, it probably would have stopped working.

Speaker A: 00:05:00

So there's a greater, as the world becomes more complex and we're more reliant on these sorts of things and these sorts of systems, does that bring with it an inherent fragility and over-dependence?

Speaker B: 00:05:10

Indeed. And it's probably the case that some, say I was relying on a service that you'd produced, Fraser, and you had used the Cloudflare server, but I didn't know that. There's several orders of potential cascading failure. And me as providing my service may not have known that it was caused by that particular thing.

Speaker C: 00:05:35

So I've got a question for Peter, probably, really, but I don't know if it's a more general question.

Speaker A: 00:05:41

I can answer it as well, I should imagine.

Speaker C: 00:05:42

Yeah, yeah, no problem. The internet was specifically designed for this sort of thing not to happen. The whole idea of the internet was to be more robust, was to decentralise things and to stop-

Speaker A: 00:05:56

Well no, wasn't the whole, are you sure, wasn't the idea of the internet to communicate?

Speaker C: 00:05:59

Yeah, but the idea was if part of it got knocked out by a nuclear bomb, if part of it got nuked, the whole rest of it would keep working, right? So it's become the very thing it swore to destroy. How are we more vulnerable to single points of failure now than we were in the 60s?

Speaker B: 00:06:19

In some way we're not, though, because this one change didn't knock out other service providers, so this was just isolated. So you've got this application level layer that we're dealing with where this Cloudflare cock-up happened. There were lots of other service providers who were completely oblivious to it, it didn't matter a jot to them. In fact, if anything, they probably got more bandwidth. But they didn't matter, so you've got that level of robustness in that you can have multiple overlapping service providers. But it's still the case that the levels below that application layer are incredibly robust, so the things that are delivering the packets around between the servers to your client, they are built on very similar tech that ARPANET was built on. And that is incredibly robust and resilient and auto-healing and things, so as parts of the network fail through hardware failure or through flooding or whatever, or cables being cut, traffic is automatically rerouted around different places. So there are differences. Fragility is not at that layer, although there are vulnerabilities to it, but fragility is at the top layer because of things. Largely, often because of human failure. Chris?

Speaker D: 00:07:41

Yeah, well I think the thing for me is not necessarily how fragile is the internet or cloud dependence on the cloud, but how dependent are we on the internet? That's the sort of thing that ... Do our lives grind to a halt at the point at which ... So even if we think this thing is incredibly robust, which I have no doubt that it is, the fact that a minor perturbation in it can cause potential havoc is I think the more worrying thing. So it's not about the robustness of the internet per se, but when it does go wrong, either in its entirety or in part, how much are we affected by it? And I think nowadays, an enormous amount. And so that's almost the resilience planning we should be thinking about is where else you might

Speaker C: 00:08:44

deliver those services, store that information. I mean, I feel like what we've done is consume the additional robustness benefits that we could have had from the internet. We consume them to make things cheaper and more efficient. And as a result, I feel like actually in terms of our day-to-day services, they are much more fragile. And by fragile, I mean much more prone to failing because of a single error. And I think a typical example might be if you're booking a flight or something. In the old days, you'd have gone into a travel agent. They would have done something with a piece of paper and possibly faxed someone. And a human at the other end would have transcribed it into their system. And when you rock up with your passports, a human reads it and they go, oh yeah, I can see you've got an H in your name where you shouldn't have, but that's fine. It's obvious that you're the same person. Whereas now, the system is not resilient to that kind of fuzziness. So if you have made a spelling mistake on your check-in, you are not going to get on that plane. If your passport does not match the spelling of the name of the person who's supposed to be traveling, you've got a massive problem on your hands. And I feel like that kind of experience is something we've sort of, in order to make things cheap and easy to deliver to the internet, we've sacrificed away that robustness. And I always feel like it is much, much harder when anything goes wrong, it's much harder to deal with it now. If restaurant booking goes wrong, if the restaurant happens to be closed, it's very difficult to find that out. If you're trying to appeal against a parking ticket, or if you're trying to get hold of someone in the council because some specific thing is, it's all much, much harder to do now. Or just trying to get a receipt from Vodafone. Or anything, right, exactly. Where in the past, these things could have failed quite a lot and you'd still have got something that would do the job. Now, it's just because of these systems being so rule-based, for what individually all looks like good reasons, we're now in a position where if one small thing goes wrong, there's almost no human on earth who can stop, who can correct it and get it right. But we all benefit from that trade-off that

Speaker B: 00:11:09

we've made. So flights are cheaper because there are less people involved in the process. So you can now fly to Europe for 30 quid, whereas 20 years ago, the equivalent price would have been

Speaker C: 00:11:19

much higher. Agreed. And I think that's the issue. But the big question is, when we're faced with these catastrophic failures, like with Cloudflare, I know it wasn't catastrophic in strategic terms, but the question is, do we understand what those risks are? And are we actually making the right decisions? And I'm thinking of things now, but it's sort of the dependence that we've built in on, to access our services through, streaming is a good example. So, I've bought a bunch of films on Amazon Prime, for example. I own lots of games on Steam. I don't actually use Spotify, but I know people who've spent a long time constructing playlists on Spotify. They are not going to have any of those things if any of those services go down. Whereas, if I have a piece of physical media, I can do what I want with it. I can be as safe as I want with that. And you might say, well, actually, it's more likely, me as a person, I'm more likely to cock something up and delete my own data than Steam or Spotify. But that isn't the point, I think. I sort of feel like the problem is, my dependence is the problem. It may well be safer, but I now no longer have any way to manage that risk. It's out of my hand.

Speaker B: 00:12:39

Yeah, I mean, yes, I think you're right. Because you've sacrificed agency for convenience, I think is kind of a high level way of putting it. But a lot of these sort of internet cock-ups occur when a process either is badly defined or not followed correctly. So, there's another good example. The Verizon BGP optimizer problem in June this year was a major, proper, low-level internet outage for a lot of the United States, where basically a small internet service provider provided some updates for Verizon to include in its DNS, which would then get populated across the rest of the internet. And nobody, all the checks weren't followed properly to make sure that it was logically sound. And these were all pushed up and everything stopped working. Like, really, seriously, a lot of things stopped working. But that's because there's some humans not done their job right. That's always going to be the case, isn't it? There's always going to be human error. But if this process could be more automated, it potentially could be safer.

Speaker C: 00:13:58

That's your classic engineering approach. The problem is that if it's more automated, it's even worse when something goes wrong. It's harder to find what's going wrong. And the results are likely to be more catastrophic. So, I mean, this is exactly, I feel like that's the, exactly the urge that pushes us towards consuming robustness. Because we always think, well, this was a bit of a pain in the ass. We can almost like kick that problem slightly further into the tail. We can make it less frequent that this is going to happen by changing something to make it more automated. But the problem is when it does go wrong, the results are much harder to deal with.

Speaker A: 00:14:34

This is exactly what has happened with my work. Because one of my responsibilities is looking after video conference rooms, of which we've got dozens of them. And we had a system where previously we used to have people who were not able to interface with it, they were locked out of it. And that kind of worked fine. But then there was a new system where everything essentially gets controlled centrally, and we can, and it's even more automated, essentially. And sure enough, it does work really well until something goes wrong. And when something goes wrong, Jesus Christ, does it go wrong? And it's exactly as you've just described. So, yeah, Peter.

Speaker D: 00:15:11

We've not heard from Chris for a bit. No, I mean, I was going to say, the cloud, to me, in conceptual terms, appeals because it is distributed storage, right? And the one way, you know, I mean, you look back, right? And you think about things like the burning of the Library of Alexandria, which may or may not have been a modern example. An actual burning, exactly. But, you know, here you have the potential, you know, devastating loss of lots of information. And we did, we have lost real, real books and plays and works of art, like, you know, the lost books of the Bible, and Cardinio by Shakespeare, you know, the lost play about Don Quixote. And, you know, there are things that have gone missing. And the reason why that is the case is because, you know, knowledge didn't used to be as distributed. You know, books themselves were expensive and, you know, few and far between. And if you lost those copies of the books, you know, look at the dark ages in Europe and, you know, the sort of only the fact that you had the sort of, you know, the Arab caliphates keeping alive classical knowledge and reintroducing that through, you know, Andalus and the Holy Land and Sicily and so on, that we rediscovered that knowledge. But that might have all been lost. And there are ancient civilizations of whom we have not a jot of information about other than a few stone buildings. And, you know, even things like stone tablets, you know, cuneiform, you know, you think cast it, you know, that's carved in stone, you think that is sort of immortalized, but it isn't. So my hope was for the cloud, that it was a technology which

Speaker A: 00:17:13

was ensuring that knowledge didn't go up in smoke, literally. We're fairly close to, I've got something I want to move on to, but before we do that, anything, Peter, do you want to come back

Speaker B: 00:17:28

on anything here? Well, just to speak up for the engineers, I think it's quite astonishing how infrequent these major outages are, given that there's so many millions of boxes involved, so many millions of people involved in running it and updating it and adding new things and build it. And it's all sort of, everything's on the internet is kind of always bleeding edge. You know, there's always some, you build something, you're going to be relying on somebody else's code, which is probably only a week or day old. It's amazing that things don't fall over more frequently, I think. And that's, I think that's, that goes back to the original, the resilience of

Speaker D: 00:18:05

the fundamental technologies. And the fact that it's a constantly live service. Yeah, constantly

Speaker B: 00:18:09

live service, serving everyone on the planet, more or less. Yeah. I mean, I think we should shoulder

Speaker C: 00:18:14

some of this burden, you know, it's not, as Peter says, the reason that we have sort of got, you know, in some cases, a lot of interdependence is because we've demanded that, you know, we want all our things to be integrated. We want to be able to, you know, have a web app that's served from a different server because it's easy to have everything in one place. And, you know, it's easy to use when you're writing code, it's much easier to use a library that's already out there and is going to be updated. And, you know, it's just, we want those things, we're not willing to bear the cost of robustness. And I think, you know, we have a responsibility to mitigate those risks, like, you know, certainly we as a company have, you know, I think we have hard copy backups kept

Speaker B: 00:18:55

somewhere, don't we? Not hard copy, because that would be a lot of printing, but we have backups.

Speaker C: 00:18:59

Yeah, yeah, I didn't, perhaps I didn't mean that. I mean, we have an external hard drive. Yeah. And I personally back, I've got a sort of multiple backup system where I have a daily backup on an external drive, and then I have a separate drive, which is kept in a different place. And, and, you know, other things, things that are very important to me, like the music that I've got on my playlists, I own, I don't, I'm not locked into a service for that. You know, I wouldn't be too devastated if Amazon went under and I lost access to some of the films. But, but I would, if I lost some of the music, which doesn't exist on the internet, you know? Well, this sort of nicely

Speaker A: 00:19:30

brings me brings us on to what I wanted to talk about. And we need to be careful here not to stray into, to territory, which we often do, which is millennial bashing, right? Oh, God, no, please. No, no, no, no. Well, that's what I'm saying. I don't want to. So, because what I want to talk about is, is the effect of this dependence has on society and individuals. Because what you're talking about, some of the things you were talking about, these are our choices, it's more convenient. And it's this sort of, and we want this, this dependence on this stuff, and this streaming, and this access to this. And I'm just wondering...

Speaker C: 00:20:08

We don't want dependence, but we're willing to accept dependence as the price. Yeah, and a trade-off is that you lose... Of having cool stuff.

Speaker A: 00:20:16

Yeah, a trade-off is that you lose a certain amount of autonomy, let's say, of agency. Because we increasingly live in these systems where we rely on the system, we have less agency, we have less input into it, but everything works better. And also, because we want it, we can become more demanding, we get used to it. I mean, do you, am I sort of, am I, am I straying into boring territory of...

Speaker C: 00:20:38

Carry on, where's it going?

Speaker A: 00:20:39

Well, I think it results in individuals who've got less wit about them, you know, and a bit more needy. He is bashing millennials. Yeah, he's going all fuddy-duddy now.

Speaker D: 00:20:49

But don't you think I'm right? I think it is true that you have less ownership over your knowledge and the provision of services to you as an individual. So if all your books are e-books, those can be taken away from you. It's more difficult to do that with your bookshelf, you're in possession of those. That service can't just be turned off. Somebody has to come and physically take hold of your books. And the same with your LPs or whatever it might be.

Speaker A: 00:21:19

But also similarly, I don't know if you're familiar, there's this app out there called Waze, right? Which I use a lot when I'm driving. Okay. But, you know, thinking back to myself 25 years ago, you know, I was using my A to Z when I was driving around London. And I myself have become a dumber person, if you like. And what about people who've never even had to bother map reading because they don't need to. I mean, I don't know, am I just being really boring here?

Speaker D: 00:21:43

I think if you come, say, there's the question of whether the dependence on the service and provision of the service builds a dependency within the individual. So like, I'm thinking with music collections, right, there was a time when you had to go and find out about music, you had to go off, find out where the record shop was that had whatever, you know, esoteric nonsense it was you were listening to, find it, buy the record. And then, you know, somebody in that record shop might tell you about something. That knowledge was kind of... It was hard earned. It was hard earned, right. And now you kind of click on something and it says, do you want a playlist of all this kind of stuff? And you're like, yeah, okay, great. So even not only does the music itself is not owned by you, but the knowledge about the meta data about that stuff isn't owned by the individual either. They don't, you know... You can't have your signal LP anymore.

Speaker B: 00:22:38

But the argument that you're dumbing down people, I think it's bunk. I think people aren't any more stupid than they've ever been. They've always been pretty stupid. But if you can just do that quick query, get an entire playlist of music, it means you can do that several times a day rather than...

Speaker D: 00:22:54

It democratises that knowledge.

Speaker B: 00:22:55

It democratises that knowledge. It means that hard work done by a small number of people to curate a kind of example of a particular type of indie rock can be enjoyed and used by many people.

Speaker C: 00:23:07

Well, I think the problem is that probably and what Chris is thinking about is that it's harder to tie that to your own personal identity. If everyone has access to the same knowledge, then there's nothing special about you knowing something because somebody else can just acquire that knowledge. Whereas in the old days, you know, you could be a big fan of the... I'm trying to think of a sort of joy division or something and know all about them and have all their albums, but everyone's got that now. So that doesn't make you special. Maybe that's a good thing. Maybe it was a bit stupid for us to tie so much identity.

Speaker B: 00:23:38

I think there's a positive trend though developing where your sort of sense of self-worth and worth within society is not based on what you know because knowledge is now easy to acquire, based on what you produce. So now the cool kids are getting into sort of using hacker spaces and creator spaces to build little projects and things. They're sort of using skills that they're acquiring doing that rather than just relying on knowing stuff.

Speaker C: 00:24:04

Yeah, I think that's a very good point. And I suspect, you know, that people throughout the last several hundred years since technological change has been something that happened in people's lifetimes have always said, oh, you know, people are losing the old skills. So, you know, in our day, people would have said, well, nowadays kids can't repair cars or televisions or darn their own socks, you know, which was something you had to be able to do, you know, 40 years before that. I had a happy evening when I was a kid darning my socks. Yeah. But I mean, I mean, that's the point is like, well, yeah, we don't do that because we don't have to because socks are cheap and they're a TV repairman and, you know, cars don't break down as much. So, you know, we quite rightly consume that, you know, and I and I think, you know, in things, other things that might happen in near future, things like learning foreign languages may turn out to be less important, you know, because it'll be easier to get simultaneous translation online.

Speaker D: 00:24:59

That to me is the perfect, the perfect encapsulation of the trade-off because everybody, you know, let's say there is no requirement to learn different languages. Speaking to an app, exactly. There's something about the acquisition of a language, which is also changes, you know, the way you think and the person you are, you know, it pushes the change into the person. And equally, you know, if you if you didn't have, if you didn't have distinctions between languages, if that was all managed and it would probably erode the existence of individual languages anyway, and those sort of cultural pockets of difference would presumably start.

Speaker C: 00:25:41

Might let them flourish. Yeah, I would have thought it's actually going to be a good thing because people don't aren't forced to learn English. And anyway, it's probably a separate podcast, but I think we're, I think we're stuck with it. No one's going to row it, wind it back. We're stuck with this technology business.

Speaker A: 00:25:53

Yeah. Very briefly, just to finish this off, I just want to think of when you've been left most adrift by a technology failure. I can kick off with an example if you like. It wasn't actually a technology failure as such, but it was when I was on a business trip about eight years or so ago. I might have mentioned this before. I was in Austria and my hotel room got, I was the most important thing that I was robbed of was at the time, my Blackberry. So it meant I had no phone and I didn't really have a way to communicate to email or I had to rely on, I don't know, desktops in different places. And I had to continue my business trip to Turkey. And the problem was, what I found was not that I couldn't, that I didn't have those things. The problem was that I didn't have those things and everyone else did. And so I just was not able to operate in this world. So not exactly a technology failure as such, but suddenly it was very, very difficult to operate and go to meetings and communicate and all that kind of stuff. So yeah,

Speaker D: 00:26:56

I mean, that's my example. Yeah, well, I think the one for me probably was the time my glasses broke when I was on holiday. And I need them for driving and for being able to see anything in the distance. And that realisation of, oh, I'm not that badly, you know, short-sighted that I can't see. But, you know, it was just a complete dependence on a piece of technology. And I ended up having to sellotape them back together again, like they basically

Speaker A: 00:27:31

look like Harry Potter by the end of it. Okay, yeah, nice. Chaps? Well, I've got the opposite,

Speaker C: 00:27:37

really. It's more of a kind of a specific non-dependence, which I have always had, and I would find very hard to give up. Which is, you know, one of the reasons I really like living in London is that if it all goes to shit, I can walk home. I'm not dependent on anyone to get me home. It takes me about an hour and a half to walk home from central London, but that's doable. And, you know, there have been a few times where, you know, there was a big blackout, I remember, in the 90s, where I was, late 90s, where I was stuck in the centre of town. I had to walk home because the tubes weren't running, nothing was running. And I was so, you know, and yet I know a lot of these people who live out in the, you know, the suburbs, or live out in sort of commuter towns, who are really, they're just, the whole evening when you're out with them is governed by the need to get the sort of last train to, you know, to Horsham or whatever. And I would hate that, you know, I really hate being dependent on other people to do something as fundamental as getting home. I can't think of an example. Peter's completely dependent on electronics,

Speaker B: 00:28:39

he would literally die. I would literally die, but I've also, I've engineered my life such that I'm, it's incredibly resilient. The closest I got was actually, I lost, the first mobile phone I've ever lost. Yeah. Was about three months ago, four months ago. Yeah. I think it fell out my pocket on a bus, or I was pickpocketed on the bus or something. And that highlighted to me a few dependencies on the phone, which I hadn't managed to find a replication for to unlock. But it recovered pretty quickly. But I've subsequently eliminated those vulnerabilities. By just having

Speaker C: 00:29:17

it implanted within his brain. Yeah. So just be like Peter, I think it's a good point. So Peter is both highly connected, but also highly robust. So we can have both. Feels like an advert for him.

Speaker A: 00:29:27

Yeah, that's good. Transhumanism, it's the way forward. Yeah. Okay. But when I can get an RFID

Speaker B: 00:29:32

chip inserted in my brain, I will because that'd be incredibly useful. I look forward to it. We'll

Speaker A: 00:29:37

have a party. Gentlemen, thank you very much. We'll wrap up there. Thank you for listening to the Cognitive Engineering Podcast. I'm Fraser McGruer, been here with Peter Coghill, Nick Hare, and Chris Wragg of Aleph Insights. Until next time, goodbye.