Why don't the trains run on time?
Chris, Peter and Fraser discuss the decisions that create problems in public services.
For more information on Aleph Insights visit our website https://alephinsights.com or to get in touch about our podcast email podcast@alephinsights.com
Transcript
Hello and welcome to the Cognitive Engineering Podcast produced by Tell Me Studios for Aleph Insights. In this series of podcasts we take a look at interesting topics and discuss what we think they tell us about analysis and decision making. I'm Fraser McGruer and I'm here with Peter Coghill and Chris Wragg of Aleph Insights and this week we're discussing the recent saga in the UK surrounding the poor service provided by Southern Railways. So Chris, can you start us off and just give me an overview of what's been happening with
Speaker B:Southern Railways? What's the issue? Yes, so this is a UK rail franchise that service a portion of the southeast of England and bring a lot of commuters into London and there has been a ongoing sort of labour dispute which has heavily disrupted their ability to bring people into London which has led to people losing their jobs and you know not being able to meet important commitments of theirs and has just generally been
Speaker A:very disruptive. Okay, so a railway service that's not working very well and also I believe one of the ironic things is that it's a sometimes on purpose the trains don't stop at certain stations because they want to make sure that they're fulfilling the obligations of arriving at a destination at their end destination on time as they just skip through certain stations sometimes so people can't get a train. So you've got a train service that's not providing...
Speaker B:Yeah, it's been very acrimonious and you know partly under without going into all the detail underlying it is a dispute around whether or not you should have doors that can be opened by the driver of the train with unions not wanting the guards on the trains to be cut out of the work of opening the doors. That's a slightly crass simplification of the labour dispute but that's one of the central issues. Okay, how did this happen? How did we get into
Speaker A:this situation? What's the underlying causes of this? What went wrong? Yeah, so I think this is
Speaker B:an interesting case in point in terms of government decision making and how so currently you know these franchises are sort of contracted over set periods of time and companies are given the rights to operate that rail service for a period of time. So I think the reason why we were interested in this particular issue is around how these decisions get made to award large contracts like this and how they sometimes go wrong. So what did go wrong?
Speaker C:I mean, has something gone wrong here? Yeah, I think this relates well to a previous podcast we made about the predicting success of any given investment or any venture and in the conclusion there was it's hard if it's difficult to measure which sounds pretty unprofound, pretty obvious, but in these big contracts it is difficult to predict how any one given service provider is going to perform over their tenure but you have quite a lot of data to work with and your trains are inherently intended to be predictable and measurable. So I think one of the failings with the way the contract for Southwest Rail has occurred is with the definition of how the success metrics
Speaker A:are defined. Are they not the same for every franchise? I would suspect not. I think
Speaker C:because these contracts come up at different times then they will learn from the previous one and so ideally they'll get better over time but this particular one has failed. So for example there was a few months ago they cancelled many crucial services at rush hour times to improve their overall percentage of trains arriving on time. So rather than address the fundamental mechanical issues or whatever that was causing trains to be late, they just cancelled the late trains that were routinely late. So they would meet some contractual constraint like get 80% of trains to arrive within 10 minutes or whatever it is. They solved that by making the problem worse for many of their customers.
Speaker A:We did produce a million tons of tractor this year, it's just one tractor.
Speaker B:I think that's a problem inherent with the awarding of these government service contracts is that you award them for a period of time and it is extremely difficult to amend those contracts or renege on those contracts. So essentially what you're doing is you are fixing a soviet style five-year plan for a particular thing. So when you set those criteria you have to understand all the different possibilities that occur and make it sufficiently flexible that if you need something extra or you want to change something in a particular way you have the capacity to to do that without being financially penalized by the
Speaker C:provider of the service. Within government for big service contracts and this is prevalent in IT service provision as much as other services, there's a sort of slightly flippant opinion that some companies will remain nameless, they have reputations for undercutting everybody, offering a cheaper service at a given rate. And then as soon as they're awarded the contract they turn around and say well actually this is completely undeliverable as is here's a list of changes and therefore then inflate the price. Government is disinclined to reject after the bid because it costs a lot of money to re-compete and its reputation is extremely damaging whereas it's easier just to under the radar increase the budget a bit. So is this, I mean as you're saying this is
Speaker A:not just with provision of rail services, this is an issue in all sorts of areas where services are being provided. Now is it, so is the issue that government in this case can and should be better in the way it organizes its research and prioritizes how it awards contracts or is it that these are problems or issues that government shouldn't be dealing with? Because for example I just, in the case of rail services I think yes government should be involved but in a different way, I don't think it should be a franchise system, I think there should be, I must admit I prefer a more kind of nationalized, centralized sort of service. But I mean so what's... Well I mean in a sense this is a little bit of a
Speaker B:defense of the faceless civil servant that so many tabloid newspapers like to castigate and sort of insight into the constraints that they operate beneath and they have to face. There is, often there are no perfect decisions but what they need to do is balance a whole set of things. So they utilize very often with these types of contract awards, they utilize a set of criteria based measures so everybody, everything has to be competed you know equitably so you can't you know so you can't give contracts to companies that you know you happen to have a relative on the board of or something. So there is this enforced criteria based approach which on the face of it looks like it should be objective and should work well but in practice often what happens is you have the criteria for what you consider good and you score everybody against but actually undercutting that on an emotional level there's the kind of intuitive decisions going on that we make outside of these criteria. So a good example is for job applications and interviews in across the civil service there are a set of criteria that you assess people against so that you can't just give it to somebody who looks like you or went to your school or whatever but having been on both ends of these processes but both as an interviewee and an interviewer it's apparent that you know these criteria while people probably faithfully try to apply them there is still an element a human element to this you know to the decision making which is I've sort of got a warm feeling about that person can't quite describe it to myself but I'm going to reverse engineer my scores for the criteria based on on that and I think this is all going on at a subconscious level I don't think people are are fixing things but they are making a more emotional judgment first and then using the criteria to establish that and that
Speaker A:presents a very interesting problem. Okay well this sounds like familiar Aleph territory because this feels like you know trying to apply and develop the best processes that we can and apply them to what is an imperfect world that something will always go wrong somewhere as best but that's no reason I have learnt for someone like myself just throw my hands up in the air just go oh well let's just employ my mate from school then. I'm glad you're learning something. Thank you, thank you. So where do we go from here I mean is that it we have we solved it?
Speaker C:Well I think I don't think we've solved it because there's still a lot more work to do and Chris mentioned that job interview is a good example it's very difficult to produce a fair and but quick and cheap to implement assessment mechanism for working out who the best candidate is but if we boil it down to sort of really fundamental say we want to buy or hire a thing to add some value and there's options for what thing we can get that could be a person or it could be a service or it could be a product. The main two factors are what are the chances of that succeeding because there's always some risk in delivery so the program might just fail to deliver what it says it's going to deliver and what the overall impact positive impact that or value that that is going to add so the the product of those two things give us a sort of expected outcome expected value and by breaking these down further it's possible then to consider where the value is and how likely it's to succeed so this is this is this is manifest in assessment processes like job interviews and ones that we regularly go through with with bids they they assess the commercial viability of the of the bid and they're saying well we've these this company has delivered bids like this before they've had success rate of this and the impact it's like we understand logically what they're proposing and the value that they're proposing is going to provide us so we can sort of get a feel for how much what they expected return is going to be but there's still as Chris says very very heuristics based and it's it's very difficult to abstract yourself from from your personal opinions of things and if you if you've had a bad bad brush with somebody but they are best candidate then your opinion of them will be
Speaker B:will be tainted and I think there's also a need to understand so you know yes you can you can talk about the likelihood of achieving success in a contract and build set criteria to maximize your chance of of making the right decision about the award of something but you also need to understand your own organizational dynamics and and you know human nature and the way these these criteria tend to be applied by people after all it is people doing them you know we're not necessarily measuring sort of quantitative data about things it's people applying judgment and and you need to set up processes that enable you to counteract this subconscious bias that's that's you know likely to to be going on and I think you've also got to understand the things like particularly for large government decisions the political factor that comes into these decision making making processes you know interjected by the ministers within departments which is you know there might be the optimal decision but if it involves a company from country x that might be unpalatable to our public and so the best the best decision may not be able to be made because the political calculation prevents that from from being chosen and so you choose a less desirable option to meet those political requirements okay I think I think we've more
Speaker A:or less covered it from my perspective maybe you don't agree I don't know Peter yeah well I think
Speaker C:there's that looking forward I mean there's I want to come at this in a different way but go on you do it looking for me that that these current processes have to be very light touch in order to be quick enough and cheap enough to implement so say you we propose a bid of for a piece of work there's a hundred thousand pounds you don't want to spend more than a few hundred pounds assessing all those bids because the the proportion that you're spending in running the competition starts to be a significant proportion of the overall value that you're trying to get out of your your your the cost of this project so it has to be like touch but that's that's largely because of the availability of reliable data and you're relying entirely on what people have said in their bid and a bit about what you know about them from other from other works but the this modern world we routinely collect a lot more information about ourselves and about other people that could be rolled into this so rather than just looking at your rather than just considering how much you like a company or you think you can trust them to give you a heuristic feeling of how good they are how likely they are to deliver a project you you could you could say well let's let's actually look at your entire portfolio and how successful you have been in projects of this size of this complexity in this area etc who are the who in your consort who is in the team look at their individual success rating so what's their individual portfolio of success of performance over their entire working life and so you can you can you potentially can get a lot more granular
Speaker A:if you can find the data so um the ministry of transport should be coming to our left and saying right sort this out for you our our our bid process you know that's exactly what
Speaker B:to be commissioned by yeah but but i think as well it's sort of um you know it's trying to give uh an understanding that um you know we live in a massive blame culture uh at the moment and every public decision that goes wrong um there's this requirement to personify the blame right and find the chief executive of some health authority or child services department or a minister and and have them held personally accountable for this and i think it's it's about viewing it as being much more systemic and you know saying that you know the civil service and government in general you know is is not is not full of blundering incompetence it's full of people who are broadly representative of of who they um who they serve and uh they are you know they are applying the same kinds of decision-making principles that that we apply and yes they can be improved but people should not be personally castigated okay uh a fair point
Speaker A:although i think people get i think one of the reasons why it happens is is just people get so angry when they can't get to work and they understand yeah but look i just want to shift focus slightly because i know ostensibly we're talking about trains but really we're talking about um analysis here of the the research process but nonetheless i just want to uh round this off just by thinking i was going to say well which country has the best trains in the world i'm not going to ask that what i want to ask is um from your personal experience what has been your best ever uh public transport uh experience could have been trains could be something else um in in could be in the uk could be another country is anyway i've got a couple but
Speaker B:anything spring to mind immediately yeah so um well not necessarily a good experience but a a slightly amusing one uh which um involved uh going on a sleeper train uh from brussels through to east east germany um and um the the the train broke down so it was a deutscherbahn train and it broke down over overnight um so there's obviously this stereotype of the the highly efficient um german transport system uh and the guard came through uh and i sort of looked out the window and realized we weren't where we were supposed to be at that time of of day uh and um said oh it's all the belgium's fault
Speaker C:okay and peter well i did a very similar trip um a friend of mine and i we traveled from um north germany uh all the way through germany down into czech uh czechoslovakia um or czech republic by that point um so down to prague and uh it was a great journey the first train the first german train i ever got was late so the stereotype has definitely blown out the water for me but when we got to the border we had to cross deck onto a completely different uh run down to train and it was it was it was like something out of a movie it was going from a sort of plush modern western europe onto this very shaky old sort of soviet era uh rolling stock and it was it had a sort of poetic um sort of nature to it and we we were joined in our little it was one of the old style carriages with the separate boobs with seats facing each other um and we were joined in our in our compartment by two complete non-english speaking uh western europeans and i think they were probably czech but the first thing they did was start babbling hello and being friendly to us and offering us loads and loads of vodka so it really felt right yeah i like those yeah
Speaker A:those old carriages that would all be familiar with um one of the one of the best experiences i had actually was in honduras when i lived in honduras where they've got this central american institution which is this private company called hetman which runs a network of coaches throughout central america and again talking about what you were saying with the german example this was kind of the opposite of that it was so well run so efficient and great food and the seats were so comfortable and because i was in a developing country i had the money to to go first class and i had this marvelous reclining chair and i remember just sleeping so well wake up in the morning in this cloud forest in the middle of guatemala and it was just a perfect experience and and i guess the other one is um sort of rather more cliche but in um japan just traveling on trains in japan is just an absolute joy and you do just feel it well it is everything you expect it to be that was my experience you know so uh so yeah so and i won't ask the boring question why can't we be like the japanese and have our trains like that so that could you know i'm not sure if we could get any value from from that discussion tens of tens of billions or hundreds
Speaker B:of billions of pounds might have something to do with it there's the answer okay right chaps thank
Speaker A:you very much indeed anything you want to add peter yes actually there is um it's not that i
Speaker C:always try to grab the last word but uh it's just i never find a way of weaving in but the the i think it's an important jump to the defense of our for you know our former colleagues in the civil service again i think the the uh to to try to explain some of the bad press that they get these the the government is going through the same learning process that big business does when it's outsourcing or hiring in services it you know big business makes big mistakes only that's much less public um so the the the learning journey that's gone through is often the same process where you know we've tried to do everything national then we try to outsource everything and we've we're now balancing back again and so there are alternatives to fully nationalized and and fully privatized you are there were different ways of carving up the service architecture so that the right parts are retained within government and the right parts are privatized and that those balance points are
Speaker A:being found in various large projects yeah that makes sense um okay great so thank you very much guys thanks as always um you've been listening to the cognitive engineering podcast um with Chris Wragg and with Peter Coghill of Aleph Insights i'm Fraser McGruer thank you as always for listening until next time goodbye
