Fraud Detection and Machine Learning
Fraser, Peter, and Nick discuss how machine learning can be used to fight online fraudsters, with special guest Mairtin O'Riada, CIO of Ravelin.
For more information on Aleph Insights visit our website https://alephinsights.com or to get in touch about our podcast email podcast@alephinsights.com
Transcript
Hello and welcome to the Cognitive Engineering Podcast produced by Tell Me Studios for Aleph Insights. In this series of podcasts, we take a look at interesting topics and discuss what we think they tell us about analysis and decision making. I'm Fraser McGruer, and I'm here with Nick Hare and Peter Coghill of Aleph Insights. And this week, we have a special guest with us, who is Mairtin O'Riada of Ravelin, and we're discussing machine learning and fraud detection. So Martin, if you can start us off and tell us about Ravelin, what they do, and then also tell us about what you do at Ravelin.
Speaker B:Yeah, by all means. So Ravelin is a fraud detection startup, I guess it's fair to say, based in London. I think part of it, I should be clear, is that I'm one of the founders, and I started with two other people, or three other people, in fact, with whom I worked at another London startup called Halo, which was a taxi hailing app in the manner of Uber. And I was the head of fraud there, and discovered that, like a lot of on-demand companies, it had a fraud problem. It's very common, any company that takes credit cards, any app, any service that takes credit cards, it's not very well known, but they tend to have these extensive fraud problems, because people either want the service, or they're able to resell it in some way. So there's a huge market of cards out there, and people will just get them and use them against these companies. And the interesting thing is that the company themselves are the people who end up paying for it. So what does that mean? It doesn't mean that the consumers are very well protected by consumer protection legislation, and for other various reasons. But if you are the victim of credit card fraud, you can ring up the bank, the bank will refund you, but through the process known as a chargeback, the company with whom that money was spent by the fraudster is on the hook. So that was the kind of problem we had at Halo, was the head of fraud, and they had this significant fraud problem.
Speaker A:And what was your background? How is it that you became an expert in fraud, and specifically in,
Speaker B:I guess, in the technology of fraud? I was an intelligence analyst before that. I was, I mean, a crime analyst, effectively. I didn't have any particular expertise in fraud, but I'd become used to kind of trying to automate some processes, you know, in the police, particularly around detection, and statistical modelling, statistically, statistical prediction of, of when and where certain crimes might happen. And it turned out with fraud, because there was such a richness of data that it was it was relatively easy to model certain characteristics of the fraudsters. And it was it was that was the basis that I started in Halo 1.
Speaker A:Okay, rather than myself asking a question, I think either Peter or Nick should ask a question.
Speaker C:Well, yeah, I was just wondering about, so when you talk about these cards, you mean, like a credit card that doesn't belong to someone? Or is it a totally made up credit card? Or how, I suppose, what's the process whereby someone is able to commit fraud?
Speaker B:Right. Yeah, I wasn't clear on that. So cards, people's credit cards are valuable commodity, because you can use them to spend money. And this is completely intuitive understanding there. Credit cards are valuable, there's inherent value to them. And you can use them to spend money on anything. So there's always been a market in trying to get other people's cards onto onto taking other people's cards and use them for yourself. And that happens in the most mundane, obvious ways. You can just look over someone's shoulder and get them. And various technical countermeasures were put in place to stop this kind of thing, such as the chip and pin that we've had in Europe for the last 10 or 15 years. But cards can be stolen in that simple way. And sometimes they're also taken from major breaches, particularly prevalent when large technology companies store people's credit cards in an insecure way. And then the next thing you know, Target in the US is because 100 million cards are on the market.
Speaker C:So they're all real cards, right? So I suppose the question is, how can you tell that you're dealing with fraudulent payment?
Speaker B:So, well, this is interesting. So to talk about the genesis of kind of this credit card fraud detection, it had the same kind of basis in automation that a lot of things do. So the first instinct to try and automatically detect fraud is to just look at it, eyeball the data. You know, back in the 90s, when people first started selling things on the internet, people started seeing that they were getting chargebacks. And they thought, well, I'll just take a look at the ones that I'm getting chargebacks. And then they see that maybe they're all associated with the same address. One fraudsters got hold of a bunch of cards, and is just using them to make purchases. So we've linked it to an address and okay, I'm going to blacklist that address. So to automate that, you just create like an expert system, a rule engine. And this is a really, really common way that you can initiate the kind of automation of credit card fraud detection. And it's still in place, it's still useful, loads of companies still do it. But you know, like any expert system and rules engine, it is a bit of a blunt instrument. It definitely has consequences in terms of false positives, because any one of us could be the victim, or not the victim, but the victim of a rule engine by just having a typical behaviour. So for example, I lived in Kosovo for a year, and tried to make a purchase on eBay. But this confounded all future purchases on eBay for the following 10 years, I was never able to convince them that I wasn't a Balkan fraudster. And that was simply because I had been put on some blacklist has used card in Kosovo.
Speaker C:Yeah, I was in Las Vegas a few years ago, and my card stopped working and Barclays phoned me up and said, I'm afraid we think someone's using your card in Las Vegas. And I had to admit it was me.
Speaker B:You missed a good opportunity there to say, Oh, that's a surprising. I've never been anywhere near. That's a good tip right there. That's from from a professional.
Speaker A:That is kind of fraud, though. That is definitely fraud. Raven does not endorse the commission of fraud. So Peter, would you have got if you've got a question?
Speaker D:So presumably, these rule engines are old hat, and they've been replaced now by more intelligent systems.
Speaker B:Yeah, that's right. And that was the basis that we started raveling on. So it's all very well, if you're an old fashioned, or e commerce company where you've got some time to interdict the sending of something from a warehouse, or you can go down and physically stop the goods from being sent. But in an on demand space, you don't really have time. And you might be getting hundreds and hundreds of orders a second. So rule engines typically will put things into three categories. One is let the order happen. Another one is stop the order, we don't want to have anything to do with it. And the third category, which is the category of uncertainty, is let a human review it. So machine learning is a really good way to narrow that review category to narrow it so that you can have less human involvement than you would, there's no need to have just a huge team of reviewers and that the human involvement effectively becomes confirmation rather than full investigation on every single order. Also just lets you, I mean, giving it the huge corpus of data lets you just reduce the number that you might have in that middle category.
Speaker D:Okay, Peter, and presumably, if you automate the process of deciding whether or not you need to review, you potentially you can then repurpose these analysts to conduct more intelligent, broader strategic investigation to look for bigger patterns and things.
Speaker B:Precisely. Yeah. So it's humans are good at things that humans are good at, which is machine learning is also very profound. Yeah. But it's a waste of time to have intelligent agents mindlessly reviewing things that you could derive statistically. And that's the kind of ultimate purpose of machine learning. If there's a statistical basis for it, machine learning is going to be better at it.
Speaker C:So what data does it look at, I suppose? What's the information that it uses when it's making a judgment as to whether it's a pass, fail or review?
Speaker B:Well, that's another interesting question, because with the rule engine, you might have 300 or so, but often far, far fewer rules that try to prescriptively determine what you think behavior might be. But with machine learning, there's no prescription at all. The characteristics or features, as we call them, are just all derived from the characteristics of the individual, the order, the card, the email, anything and everything associated that you might get as a bit of information. So things that seem non-intuitive, the ratio of consonants to vowels in an email address, the amount of time that someone spends on a site, whether they go straight for the high value stuff, all of these, you know, some of them can be encapsulated with rules, but then you need to make a judgment about what weighting you should give that and what score you should have. And we don't take that. We just say, right, we think it might be relevant. We're not certain. The machine will tell us by giving it a corpus of examples and labeling them as either fraudsters or genuines, the machine is able to pick out and say, well, these are the characteristics that were important to a fraudulent order or
Speaker D:fraudulent customer. Presumably because you're letting the machine just decide what it statistically reckons is important in these factors, there may be no obvious logical reason why the proportion of vowels in an email address gives you a higher proportion, a higher chance of fraud.
Speaker B:Exactly. And that leads to another really interesting problem with machine learning. This is one of its great disadvantages is that there's a diminished explicability compared to a rule engine. If a rule engine triggers, a rule triggers, you get a rule. You know why it's triggered and it's a yes or no. It's a binary gate. With machine learning, particularly on certain types of algorithms, neural nets in particular, very hard to interpret the output. You know what the outcome is, but you're not sure how it derived that.
Speaker D:So it'd be interesting. You can imagine interesting court test cases where, oh, I tried to buy this thing. And as a result, I suffered an injury. I'm going to sue you. Taking really difficult to unpick the audit trail of why that purchase was denied in court for humans to understand.
Speaker B:Completely. And in fact, it's always been a challenge for Ravelin, as people selling kind of machine learning technology to effectively sell this kind of lack of understanding compared to what they had before. To the extent that we explicitly favor algorithms that have some amount of explicability over ones that maybe have a slightly better performance, but have no explicability. Because we found that in this kind of great move to machine learning automation, people still really want explicability. Whether or not it helps them.
Speaker C:And I think that sells us something very interesting about how humans reason, which is different from an idealized way of reasoning, which is that we tend to think in terms of processes and systems and cause and effect. And if something happens, we don't really feel like we understand or can use the connection between that thing and some other thing until we found what the causal link is. It took a long time when statistics showed that there was a connection between smoking and lung cancer. People didn't really feel like they quite believed it until they understood what the mechanism was. But presumably your algorithm, like a lot of other algorithms, just don't care. I mean, the fact is these things are connected. So one bit of data explains another piece of data statistically, and that's enough to say, to inform a decision.
Speaker B:Absolutely. In as much as a machine does or doesn't care about anything, they absolutely don't care about the means by which it reaches a conclusion. But people really, really do. And for that reason, we have to acknowledge that. I think as time goes by, people will be more and more comfortable, perhaps. I mean, it's a thesis that people will get more comfortable with handing over more responsibility to algorithms. For example, I don't need to know how a car works. As far as I'm concerned, it's magic. But when people first got into them, I imagine that they were very, very concerned with the mechanism and means by which it propelled. Checking under the bonnet. Checking under the bonnet. Yeah, for little people.
Speaker C:I mean, so I guess it actually has another advantage, which I hadn't really thought about, which is that it actually makes it harder to game. It's harder to play against something if you don't know what it's looking for. So, you know, if you realise that it's something to do with addresses, then you'll look for ways to fake your address. But if you don't really know how it's working, it's harder to do that. Well, I think something that Peter and I were wondering about was, could you imagine someone developing an artificial intelligence that would be designed
Speaker B:to break Ravelin's system? I can 100% anticipate that. I don't think it's terribly likely, because I think fraudsters, it's not likely in the near term. The probability approach is one in the long term, because, you know, there will always be countermeasures and fraud is a lucrative career with little enforcement and almost no consequences. So it's a lucrative thing for people to do. But it's one of the great advantages of machine learning over a rule system or an expert system. Fraudsters will find those edges, you know, a prescriptive edge a fraudster will find and find an exploit, either by just, you know, waiting enough time or by changing the address
Speaker A:or doing some simple countermeasures. Okay. I just want to take it in a slightly different direction to talk about, we've talked a little bit about, for example, one of the problem, one of an issue is if there's no explanation that humans don't like that there's no explanation, but flipping that around a little bit, what, what are the human inputs? What do they still have to be the input, the inputs into the detection? And what could you foresee that would never change?
Speaker B:So there's, we're nowhere near a point where you can have no human interaction, yes, not in fraud detection, and not, not in the kind of space that we're currently in. If the most important thing we have at the moment is feedback for whether we got the decision right or wrong. It doesn't have to be on every single decision, but it has to be on a decent representative sample. So every decision that Ravelin makes, we'd like to get it, not everyone, but we'd like to get as many as we can, the confirmation of whether we were correct or incorrect, because it ceases to be accuracy metrics are, are interesting in machine learning because you could say, you know, if we let every single person through, we would still be 99% accurate in most cases, you know, it just that, that 1% is, is, you know, causes a lot of trouble for, for, for our clients.
Speaker C:And if you didn't let anyone through you, you'd stop all fraud. We also have stopped a hundred percent of fraud.
Speaker A:Sorry, I'm still missing. So what is it that humans still need to do?
Speaker B:So the review is still, is still a thing. The people who have been analysts before still have to review, but this, this time tends to be saying we were right or we were wrong. The decision the machine made was correct or incorrect. And, and the ones that it's incorrect on, we package up and say, we, we look at those too and reincorporate those into the learning set, right? These are, these, these ones weren't as fraudy as we thought. The ones that we were wrong on or well, right and wrong can mean you either said it was a fraudster and it wasn't, which is a false positive, or you said it wasn't a fraudster and it was a false negative. So those go automatically into the learning and retraining feedback cycle. And it's really important for the analysts to, to feed those, to give that, give us that feedback so that, so that we have better examples, ever better
Speaker C:examples to train on. Presumably you also had to decide, you had to make some judgment about what information you needed to gather and to give to the machine. Exactly. And, and because presumably there's a whole, there's also tons of information which you could be using, but which you just judged to be irrelevant. So, you know, I, I'm only guessing, but you would imagine the, I would guess things like the time of day might be important. Yeah. You probably, you, you might imagine saying, well, the number of seconds that have elapsed since the end of the previous minute isn't, or, you know, the, which maybe, maybe postcode is important or, you know, but, but some things you might decide weren't. So is that something that's becoming less and less of a problem because there's fewer constraints? You can just give as much as you possibly can to the
Speaker B:algorithm. You can give as much as, as you possibly can, but it, but it's still a real need for a person who understands deeply what, what the problem space contains or what the, that industry, the kind of features that are important intuitively in that industry, because otherwise we might just miss really obvious things. We can say that like we, we might be collecting 400, 500 features on any given order, but not one of them might be the most important one. So there is definitely scope for that kind of human expertise, top level human expertise. The person who once would have written the rules is now saying these things are important, but not being as prescriptive about the way that they're important and what weight we can offset that to a process that's better at doing it than, than, than you are or that one is. But, but we still need that feedback because he might say, or she, you know, would often say that something that we hadn't thought of at all is really
Speaker C:important. Yeah. And so I suppose in future, you can imagine this sort of thing, just being the norm for a lot of industries. So for example, Peter and I come from a defense background. You come from a crime background. You can imagine an intelligence analyst who's looking for warning signals of say a nuclear test or something, not needing to actually interpret those signals themselves, but saying, okay, these are the kinds of things we might look at and the machines will do the rest. They'll, they'll say, okay, well actually we've looked at all of the past data and we think there's a, you know, probability of 11% of a test in the next week. But the human will still be needed to point it in the right direction to, to give it,
Speaker B:to decide which data streams to look at. I completely think so. It wouldn't be any good trying to use our machine, our trained machines to predict the probability of a nuclear test because the features would be totally ridiculous. Would the architecture work? I mean, I mean,
Speaker C:can you just plumb in, couldn't you just plumb in different data? Completely separate set of
Speaker B:features. I, I think it probably depends on the algorithms you're using, you know and it probably depends what you want the output to be, whether you want it to be, you know, a prediction or a classification or whatever. I think it probably depends more on that. You could certainly, you know, the techniques are, are generalizable, you know, that's, but I don't think, you know, our, our software would work particularly well. It'd be the, but the, the underlying algorithms,
Speaker D:the statistical machine learning methods are probably all the same generic ones that are
Speaker C:repurposed elsewhere. I'm certain of it, yeah. I mean, my, because my thought is whatever your analysts are doing when they're saying, you know, is this correct or not, they must be using some sort of information. So in principle you could, you know, then just take the information they're using to make that human judgment and plug that in. And I, and I wonder if we're just not being a bit too generous to, to humans here. And if in fact, you know, actually one day we really won't
Speaker B:need humans to make judgments about things. So in, in the case of this kind of payment industry, there are other mechanisms for feedback, you know, and to some extent humans appear to be divorced from them. So merchants might get something called a chargeback file from their acquirer, which is a bank for merchants. And that has a list of, that has a very good list of these chargebacks that are the very high probability that they're, these originated with fraudulent accounts. And you could use that to train instead of having any human reviews. In fact, some of our clients do that because they don't have humans involved in the process, but behind the scenes, you know, every chargeback starts with a person ringing their bank and that person is someone human involvement somewhere further down the line. Now, of course I can anticipate a different system where there's just no human involvement in it at all. And the feedback in a particularly pure system where the feedback is constantly coming in and reinforcing it, a really good application for machine learning that would be. But to the extent that you have we're in the real world and that, you know, we need quality feedback.
Speaker C:So at some point someone's got to say this was a fraud. Someone has to say this was a fraud. That's a real life thing has to happen.
Speaker D:So what is it about, what are the features of fraud detection that make it easy to automate, that make it vulnerable to automation with machine learning?
Speaker B:So a couple of things, you know, the fact that fraudsters are to some extent predictable, that fraudulent behavior is some extent predictable. There's also been, you know, 10 years, 12 years of people training themselves to be decent fraud reviewing analysts and machine learning is very good at things that people are good at in bulk. So we've got a huge corpus of a body of work that's already been done where we have people trained who are ready to train the machines without really much interruption to their daily lives. So there's little with little change in their process. They can train a machine to do it for you. You know, unspoken part of this is that they're training themselves out of jobs in some cases, but humans should do what humans are good at.
Speaker D:Presumably the availability of data, which includes that training set that you're talking about, but also this live stream of data.
Speaker B:Yeah. So people are, yeah, I mean, more purchases are being done on the internet, either through apps, services, web. I mean, the UK in particular is heavily, heavily, by far the most comfortable country with doing internet purchasing. So the volume and richness of data is on a kind of curve upwards. Not sure it's linear. It might be.
Speaker D:So using that as a sort of understanding model for other industries might be vulnerable to machine learning. What sort of immediately stands out or what other areas are aware of that using the training to be?
Speaker B:I think so. I mean, a lot of financial, the financial sectors is possibly in danger. Compliance in particular, which is just kind of rules. Insider trading. These are all ones that seem to me to be very vulnerable to the similar types of techniques where you have rows of people looking at trades, you know, then we talk about other things that are, you know, heavily process led. I mean, this isn't necessarily machine learning, but, you know, certain types of, you know, bookkeeping, accountancy. I think they're really.
Speaker C:I think the one we always we always seem to come back to is estate agents. Oh, I think he doesn't want them to be replaced by a machine. Yeah.
Speaker B:An unthinking, hard nosed automaton. Self-interested automaton. Well, that's estate agents, but what about the machines? That's my point.
Speaker D:But I think there's all, I mean, I think there's people applying machine learning to other sort of predictable, predictable industries. So, for example, manufacture. If you're a few, if you're a production line, there's very, lots of data available there, lots of things you can measure. You can apply machine learning to monitoring when things are, when your systems are about to fail, because there may be, may be predictable cues that tell you when things are, when parts need replacing, tolerances are starting going out, noises are appearing in the wheels. So you can apply it to anything that's got lots of data like that. Yeah.
Speaker B:Monitoring, I think maybe is the kind of, the meta characteristic of a lot of these things that would be, that are particularly, you know, vulnerable. I mean, monitoring is certainly the function of the fraud reviewers as, as it is for compliance and trade reviewers in the, in the city. So there's definitely that monitoring that, that monitoring, I guess is the, I call it the meta characteristic of these industries.
Speaker A:I, I think we need to wrap up there. And, but this, it's nothing to do with machine learning, but there's two questions I want to ask. Yeah, go ahead. But first of all, is what, do you have any personal feelings towards fraudsters? That's the first question. Second, I would have thought, and related to that, that you're fantastically placed to, to commit to be a fraudster yourself, aren't you? And are you, have you ever been tempted? Well, not have you ever been tempted, but more or less you would, yeah, someone with your set of skills and experience, you'd be perfect for this, I would have thought. But anyway, so what are your personal feelings towards fraudsters?
Speaker B:I really have quite a deep dislike of fraudsters, although they pay my bills. I mean, that kind of slightly indirectly, indirectly, but they, you know, the fact that fraudsters exist, you know, is, is the kind of bread and butter of my company. But I have a moral contempt for them. Well, it's not just moral, it's the idleness of them, you know, the predictability, idleness and stupidity.
Speaker C:You can make an Excel spreadsheet, trap them, you know.
Speaker A:Okay, so what makes a good fraudster and what makes a bad fraudster?
Speaker B:A really good fraudster is the unpredictable fraudster and the ones, you know, the ones that I can see have, you know, gone out of their way, probably automated the process themselves. The Wolf of Wall Street kind of fraud. I haven't seen it, but yeah, they're super high level fraudsters. Some people put an awful lot of effort into very, very low level payment card fraud, and I do not understand it. Eventually, you just think, why don't you go and get a job and earn money?
Speaker C:It actually becomes easier just to not be a fraudster, but to actually just have some money on a credit card.
Speaker D:Maybe there's another podcast in that, maybe the economics of fraud and when it becomes more costly to conduct fraud than it does today.
Speaker C:It's a bit like, I mean, we've done a previous podcast episode about signalling and, you know, sort of to make the signal really convincing, you just, the easiest way is just to be that thing. Yeah, yeah.
Speaker A:And so, and then I presume then, so that my second question, if you would be a good fraudster, presumably you would be a very good one.
Speaker B:I would not be because although I constantly bemoan the fact that there is no enforcement of, you know, in this kind of payment fraud space, or little or no enforcement. I mean, there's, there's some, and that the consequences, even if your code are pretty minimal. So, you know, all the incentives are there. I would still be way too scared to do it because I am.
Speaker D:Because Ravelin is so good.
Speaker B:And well, that aside, Ravelin doesn't prosecute. Just, it would really just tell you to not take the order to block the customer. So, you know, the consequences are limited, but I still would never do it. I'm too scared to commit a crime.
Speaker A:In a technical sense, you would be well placed.
Speaker B:In a technical sense, you might even argue that Ravelin's incentives are aligned with the proliferation of fraud. But of course.
Speaker D:There's no dark underground department of Ravelin, which is actually conducting loads of low level fraud. No, there isn't. I categorically state that. There we go. And you can neither confirm or deny any accusations. Fraud is a crime. Okay, don't do it kids. Don't do fraud or drugs.
Speaker A:Let's wrap it up there. So, as always, thank you to Nick Hare and Peter Coghill of Aleph Insights for joining us on the Cognitive Engineering podcast. But an extra special thanks to Mairtin O'Riada of Ravelin for joining us and providing us with an insight of his expertise into what you do at Ravelin and into machine learning and fraud detection. So thank you very much, Martin. I really enjoyed that. Thanks, everyone. And until next time.
