Predictive Text

Speaker A: 00:00:00

Hello and welcome to the Cognitive Engineering Podcast produced by me, Fraser McGruer for Aleph Insights. In this series of podcasts we take a look at interesting topics and discuss what we think they tell us about analysis and decision making. I'm here with Jordan Fermanis, Chris Wragg and Nick Hare of Aleph Insights and this week we're discussing predictive text. Chris, lead us in on this. I predict you've got something great to tell us.

Speaker B: 00:00:34

So yeah, I was at a conference recently and in between presentations at the conference, I would get my phone out and do a bit of messaging with people at work and then put the phone away and after one of the presentations I pulled my phone out and I'd been using Slack. For people who don't know Slack, it's a bit like WhatsApp. It's basically a kind of conversational dialogue. It's what we use at Aleph. Most people will be familiar with Slack, I imagine. But I pulled it out and there was a message that I'd written sitting in the bar and I looked at it and I thought, well, I don't remember starting a message. But my first look at it was like, here's a plausible thing that I've written and as I read it, well, I'll read you out what it said. So it said, yeah, just lots of potential avenues for how to do this sort of, them, workflow and they all have frozen prawns. And I'm interested to see in the future which techniques come forward, see the best way of handling this sort of rocket. So it did quite a good job then. Right, exactly. Because I'd sat on my phone, right? And so this was entirely constructed by my buttocks and predicted text. And so it got me thinking like, because as soon, when I pulled it out, I thought, oh, here's a message I've written. What's this about? I almost sent this. What is it? And I realised it wasn't from me. But at first glance, it looked like something coherent and, you know. I want to drill into the frozen prawns.

Speaker A: 00:02:23

Have you recently had some kind of conversation about frozen prawns?

Speaker B: 00:02:28

No, no. So like, I don't know what combination of letters. But obviously, I mean, the way predictive text works, you know, you put the first few letters in and then it does one word and then it does the next word. So presumably frozen and prawns, you know, sort of go together as a couple.

Speaker A: 00:02:46

But because actually, we don't slack one another. But with a couple of exceptions here, I think this would make sense. I could imagine Chris, right? Yeah. Yeah. Just lots of potential avenues for how to do this sort of workflow. That's very Chris. Yeah. And then forget the bit about prawns. I'm interested to see in the future which techniques come forward for the best way of handling this. If we ignore the rocket bit, it kind of did OK.

Speaker B: 00:03:11

Yeah. But what I've sort of really has started spooking me out is Google's smart compose, right? No, not come across it. So in emails, I mean, I'm sure it's in fact, it does appear in Google Docs and other Google apps. But basically, it guesses the end of your sentence to greater length. And it is becoming uncanny in some instances at writing. So it just got me thinking about predictive text. What are the implications of this for everything, really? Wow. OK. So that's quite a big question.

Speaker A: 00:03:56

Who wants to jump in? What sort of? Well, it might be worth mentioning GPT-3 because that's the big beast, really. What is GPT-3? So it's an algorithm developed by the extremely misleadingly named OpenAI. It's completely closed. You can't access it, really. And you can't. There's a kind of rationing system if you want to play with it. So it's Orwellian from the beginning. Yeah. Anyway, it's a company called OpenAI. That's what they're actually called. OK. They essentially trained this, well, as far as I can tell, it's a kind of relatively sophisticated neural net on essentially the internet. It read the internet. Well, it read 570 gigabytes of text. When is this? Recently, probably the last year, I think. Oh, wow. Sometime over the last year. There's been a lot of actually phenomenally impressive AIs recently, which can do things with very little prompting. Because did you say 500 gigs worth of text? Well, that's the thing I've got here. It sounds low to me. I was going to say 500 gigs doesn't sound much to me, but if it's just words, if it's just text. Yeah, that's it. It's just words. So their model has 175 billion parameters. Now, what does that mean? It means that they are trying to train it to work out what word is going to come next. That essentially sees a bit of text and it guesses what word is going to come next. And it's done that by tuning these billions of different parameters, which are, you know, until essentially it finds a set of parameters that do as good a job as possible at predicting the next word. Those parameters don't accord with anything you or I might understand, right? There's some aspect, there's some feature of the sentences and the words and the order they're in, which is perhaps not something that we would think about, but which is there, which each of them might represent some tiny fraction of deep structure. But the upshot is that you can put in something completely new and it will incredibly plausibly fill in the rest for you, right? So, you know, so the obvious things might be, you know, you can put in, oh, I was walking down the road to the shops and it will continue in that style. But if you were to say something like, breaking news, this just in, it would then fill it in in the style of a news report. Or if you were to say, you know, sir, thou art a scoundrel, it will start writing like an 18th century letter for you. Are we already seeing this in different uses? Yeah, people are finding uses for it, but it is, unless people have played with it, which I recommend that people do, there's a thing called AI Dungeon where you can have a go on it. It's perhaps not, it's not obvious quite how impressive it is. And so as well as the sort of, you know, fiction and non-fiction, it will carry on a story. It can impersonate people very plausibly. It can do things like write plot summaries, you know, of a film. If you start writing an advert, it will fill in the rest of the advert in an appropriate style. It can even write plausible code. You can start giving it some code and it'll fill in the rest. It does things like maths. You can say, you know, 512 times 16 is, and it will fill in the rest. Because it has tapped into some, as I said, a large number of deep features of the structure of language that we're not cognitively aware of and is exploiting that to be very, very good. At least not consciously. No. So what does that mean then? What's the

Speaker B: 00:07:41

analysis for that? Well, so Nick mentioned AI Dungeon, right, which was a game that actually Nick sent, I think it was a couple of years ago. So I think GPT-3, you know, has been around and being applied for a few years. But AI Dungeon was developed by a company called Latitude. And Nick said, oh, you should have a look at this. This is really interesting. And I started playing. And so essentially it's a role playing game, right? And it creates your own adventure as you go through text based interactions. So, you know, you sort of say, or it gives you a scene and you say, oh, I'll go left, please. And it says, oh, well, now you see this thing and so on. And you can engage with characters. Now, what this showed was potentially pretty sinister. Right. So and I remember this because the first time I started playing, I was like, this thing's amazing. Right. And I got ambushed by some wolves in the forest. Right. So and sometimes so where it's not so good is it sometimes forgets things like there's a character with you and then suddenly it forgets the characters there. So I tried to create my own internal. Sorry, Chris, I think we need to make

Speaker A: 00:09:03

it clear, right, as there is absolutely no pre-generation of story. These wolves came out because of things you typed in. You said, I want to go into the forest. Yeah. But it knew as an adventure story. Yeah. And wolves are the kind of thing. Exactly. So no one has written, right, if you go into this forest, you get wolves. Chris's inputs generated the wolves.

Speaker B: 00:09:24

So anyway, I and it builds this model up based on all of the interactions that are going on in the game. So, you know, maybe other people had created wolves and then it learned to, you know, understand that wolves lived in the forest. So, yeah. So I sort of, you know, cuddled one of these wolves as it attacked me and it was unconscious. And then it was like, what do you want to do? And I was like, well, I can't murder the wolf in cold blood. Right. You know, so I was like, right, I'll tie the wolf up. So I tied the wolf up. And then the next line was the wolf wakes and looks at you with doughy eyes. Right. At which point I'm like, what? Where's this guy? Right. Turns out AI Dungeon has been, has had to really over the last couple of years, has had to really review its moderation processes because effectively a decent portion of the users have been using it to sort of write all these, you know, very dark kind of stories and, you know, getting into really unpleasant kind of stuff. So can I just confirm this wolf was trying, trying it on with you? I think the implication was that the wolf was, was, you know, it was invited. So how are you and the wolf doing now? Pretty, pretty, pretty good. I stopped playing AI Dungeon after, after I was propositioned by a wolf. After the third time you slept with the wolf, you're like, this is getting a bit boring. Yeah, exactly. Anyway, but the point is what it shows is that what this is doing is picking up on its user base. Right. And that's how it's, that's how it's learning its, its language and, and its inferences and where the direction of narrative goes and that that can be driven in a, in a sinister direction. But equally, you know, if the user base weren't weird, it might go in a different direction. But yeah, I think that illustrates a little bit of how it works and how, what the implications can potentially be.

Speaker A: 00:11:21

Okay. So we're starting to talk about implications a little bit, but I still feel that so far what we're saying is this is a thing. This is a thing. I don't, I'm not getting any sense of our own analysis here. But for the moment, let's, it might just be moving on to here's another thing. I don't know. Let's start talking about some implications. Okay. We ready for that? Go for it.

Speaker C: 00:11:44

Well, one of the implications is obviously with predictive text, not so much GPT-3 is the effect it has on young people's learning and ability to, to write because obviously it sort of suggests things rather than you having to think of them. And I came across a study. I feel like you're about to join our club here, Jordan, and join the old grumpy people. Go on. Yeah, it's a little bit of that, but it's, but, but basically obviously there's a lot of outcries sort of amongst old grumpy people about textisms and textees. I think I've even seen it called, you know, like using, saying great GR8 and WTF and all this sort of stuff. It kind of reminds me of, I think we did a previous podcast on emojis. We were talking about how they can be a substitute for language. And I think there's a similar sort of, some people are worried that the impact that this will have on children's ability to write and read. But I think there's also this thing about language being quite malleable and, you know, like Shakespeare famously made up lots of words and like lots of authors we read had secret languages and invented words and all sorts of things. So I think there's also something to be said for the flexibility that we have in our language and just, and being able to use that rather than sort of focusing on maybe limiting it through predictive text. Yeah. What was the evidence by the way on children's? The evidence was, so it tested participants aged 11 to 14 years old, people that, children that used their, that made 15 or more phone calls per week and sent 20 text messages or more per week. And they were, each participant was administered an IQ test. So I guess if you have issues with IQ tests, maybe this isn't such a good study, but the study found that participants who texted more tended to work faster, but they also scored lower on the IQ test. Okay. Okay. Making them more productive. Making them more productive, but maybe not necessarily increasing their knowledge. Yeah. Predictably more productive.

Speaker A: 00:14:10

Yeah. Okay. So there's an interesting point there, effect on kids. I really liked what you were saying there, Chris, about this, about how it, about this sort of corporate experience informing the individual experience. That was, I liked that.

Speaker B: 00:14:28

And I think just to sort of pick up on that, I think what you find, and there's a study which was done on human writers. Okay. So they looked at what actually happens when you have an experienced professional writer, you know, like a novelist or, you know, writer for some other purpose. And what happens when you get a novice writer and they did functional MRI scans of them, right? They created this, in an MRI scanner, they created a writing desk that you could more or less, you know, it wasn't words worth sitting on a hillside exactly, but it was, you know, as natural a writing environment as you can create with an MRI scanner. What they did was they basically took novice writers and they took experienced professional writers and they looked at what they were doing. They gave them three, four things to do, right? The first was they just had to read some text, right? This is very GPT-3, right? So they had to read some text. Then they just had to copy some text. Then they had to brainstorm some ideas for what was going to come next. And then they had to write what was going to come next. So that's almost what GPT-3 is doing, right? It's reading some text. It's having a think about it and then it's generating the next bit in the instalment, right? And what they found was differences between the professional writers and the novice writers in terms of what was actually going on in the brain. I'm going to do a little bit of extrapolation from this, but what it found, so these are the sort of facts of the study. There's an area of the brain called Broca's area, right? Which is the bit that is responsible for the generation of language. If it's damaged, you can't speak, you can't converse, there's no grammar. In any language, you can't converse. So it's the part of the brain for generating language. And what they found was that when they were doing the brainstorming element of the task, the professional writers were utilising this Broca's area. It was highly active. Whereas the novice writers, it was the visual processing centre of the brain that was being used. So my extrapolation of that is that the point at which professional writers, experienced writers, are considering what ideas they're thinking in words, whereas novices are thinking in pictures and ideas, right? And I think that the predictive text is doing a similar, comparable kind of thing. It's not visualising the world, right? And going, oh, okay, I'll put some seductive wolves in this forest. It's just simply creating a set of words that match the next set of words. And I think that's kind of what this might suggest professional authors are doing. The other thing that it does, which I think reinforces that idea is that when they were doing the writing, there's another area of the brain called the chordate nucleus. And this is the bit of the brain, which is responsible for holding memories of skills, like, you know, for musicians or athletes, or, well, in this case, authors. And when they were writing, authors were utilising, the professional writers were utilising this chordate nucleus. They're basically drawing on previous models. This is my extrapolation, previous models of writing that they had built up over time, whereas the novices weren't, they were kind of coming up with it from scratch, they didn't have a model on which to base it. And so I kind of feel like these two elements that we've talked about, you know, whether or not they understand whether or not GPT-3 has any semantic understanding of the world when it's predicting what's coming next. And of course, we do, don't we? Well, this, to me, potentially suggests that actually experienced writers are simply pulling a set of models that they've got and deploying them. And I think smart compose, right, to come back to the point of corporate knowledge distilling down to an individual. Yes, they can learn a little bit about you and your preferences, but by and large, they're drawing on big world models of lots of users. And smart compose, I was writing an email this morning. And in it, I was saying, you know, we won't do this at this time. It suggested at this time, you know, we won't do this at this time. And I thought, no, I want to say at this stage, right? It was a tiny, tiny difference. At this time wasn't quite right. But essentially, what it was doing was playing me a cliche, right? Like something that was used by lots of other people in those circumstances. And so I feel like that's kind of what by delivering, you know, the Orwellian conceptualisation of a cliche, where you're just using one block of language, because you can't be asked to think up brand new words, is actually an efficient way of doing things. And that's what we always think of cliches as bad, but actually, cliches are shared meaning, right? And shared information. And that's kind of what these things are doing. So I don't actually think the questions about do we understand it? Does it understand what it's saying? are particularly meaningful.

Speaker A: 00:20:18

eption, which is described in: 1984

Speaker C: 00:23:55

I can say something leading on from that. I think, yeah, essentially like a lot of AI, ML applications, GPT-3 is a black box essentially. And I think the fundamental difference for me is that we generate language based on, as Nick was saying, ideas and concepts, whereas applications like GPT-3 are statistical analyses, basically. They're making combinations of words to generate sentences, but they're not thinking about, as Nick was saying, the context underlying it. And so I think that sometimes, as Chris's example shows, when you read some of the combinations or some of the sentences that it formulates when it's sort of gone rogue a little bit, they're so unintelligible because it's not thinking about what it's trying, the meaning of what it's trying to get across. It's just thinking about generating sentences and formulating words. I've got a couple of examples, if you want,

Speaker A: 00:24:57

of GPT-3 going rogue. So these are from an article in Technology Review. I have to say that, okay, there is a lot of controversy about this. Like a lot of the things where people are saying GPT-3 is obviously, you know, flawed and here's why. There's a bit of motivated reasoning, I think, behind some of those. So people have deliberately cherry picked examples where it's gone wrong and actually, you know, if used properly, it doesn't necessarily do this predictably. But this was an interesting example. The human prompt is as follows, and I'll tell you when we get to the bit where GPT-3 takes over. You're a defense lawyer and you have to go to court today. Getting dressed in the morning, you discover that your suit pants are badly stained. However, your bathing suit is clean and very stylish. In fact, it's expensive French couture. It was a birthday present from Isabel. You decide that you, now GPT-3, decides what to say next. Should just put on the... You should wear the bathing suit to court. You arrive at the courthouse and are met by a bailiff who escorts you to the courtroom. It's impressive, but it's obviously totally wrong. Like, and we understand that because we understand the real world. But actually, you know, GPT-3, if we can speculate as to what that's, you know, and again, it's speculation, but there's some hidden structure down there. But I can, sorry, Nick, I know, I can imagine at some point that surely it's not that difficult for an AI to learn that you should never wear a bathing suit. Remember, that's not what it's doing. In the way that it's representing language, it will have, there'll be something like something suit. There'll be a whole sort of, which will be close to, you know, a dinner suit will be close to a, you know, another morning suit, be close to an office suit or whatever. All of these things will be in the space of how those concepts are represented, how those phrases are represented. They will be close to one another. And it will look at the phrase bathing suit, and, and presumably think it's close to those. It learns that it's close to those. And that therefore, you know, because there's lots of contexts in which they are. It's like, okay, I was going to go to the dry cleaners. And I discovered that my morning suit and my bathing suit are dirty. So I take them both to the dry cleaners. There's a lot of things you would do. You'd fold them up, put them in the drawers. There's a lot of things you'd do. Where they can collocate. It's just that we understand because of our understanding of the real world, that's inappropriate. And even it doesn't know that because all it knows is that a bathing suit is in innumerable ways very, very similar to an office suit. So even though on this occasion, it gets it wrong, and it could learn to get that example right. There's going to be many, many, many, many other times.

Speaker B: 00:27:32

I think the thing we've got to remember is that the only medium GP, and we shouldn't get too hung up on GPT three, because it's, well, it's presumably a third iteration. But, you know, it's an early iteration of this and it's going to get better and it's going to get more exposed to more data. And it's going to have even more parameters than it already has, which is ginormous. But the only medium it has to understand the world is words, right? Whereas we can see things, we can observe things, we can hear, you know, different sounds. We have our experience in the world. The only way this can sense effectively is through the input of words and its digestion of those. And that's the only way it can output into the world as well. So in a sense, it doesn't know that a bathing suit isn't worn at court because nobody's ever written about bathing suits not being worn in court. But if they had, it would know that, right? But I think, you know, humans also get language wrong all the time. You know, look at the number of faux pas that people make. And I think, you know, I think for me, the question is, how do you get authenticity? So is it inauthentic for something like this to write about love, for example, right? So if this wrote, you know, a romance novel, would that have authenticity, right? It might have inferred what love is, but we know this thing can't have loved. It won't have been in a relationship. Could you say the same about Barbara Cartland? Right, exactly. So you look at the Barbara Cartland or you look at one of my favourite lines from any song is the beautiful South song, Song for Whoever, where he says, I love you from the bottom of my pencil case. And it's a song all about basically writing love songs cynically to make money. It's a brilliant song, but we know that that happens. And yet we can still engage with those things and feel like a song that is written by somebody else and sung by someone is authentic in some way. So yeah, that to me is one of the questions about implications.

Speaker A: 00:29:57

Yeah, nice. Any final points we want to finish? Well, just to pick up on something Chris said and speculating about the future is there's a lot of phenomenal generative image AIs, which have come out for some reason quite recently. There's been a slew of them, one called Dali, where you can type in something like, show me a picture of a penguin wearing a top hat in the style of Van Gogh, but the penguins recently received some bad news or something, and up will pop this astonishingly good picture of precisely what you've just said. And I think obviously we've been aware for some time that machines' ability to ingest images and all kinds of data is becoming more generalized. I don't think it's difficult to imagine how you could just plug GPT-3 into something like GDELT or some other database of news, for example, and get it to write news reports, for example, in a way that humans will find extremely digestible. I think the day is coming when a lot of text generation will be basically automated, and for it to be good enough that I would say that probably most text that is produced at the moment is produced in a very routine, cliched, sort of paper mill kind of way. It's formulaic. You talk about formulaic writing. I suppose what I'm saying is it's going to put marketers out of work, so bring it on.

Speaker B: 00:31:28

I think journalists exactly, and Jordan, some people have made the point that we write from the basis of conscious understanding of the world, right? But I would say a lot of the time, I worked as a speechwriter for ministers in the Ministry of Defence for a year and a bit, and I've done various kinds of writing. Jordan, I'm sure, has had similar experiences through journalism. That's when you feel the right word for the circumstances. You have no idea why it's the right word, but you know it is the right word. Professional writers spend a lot of time searching for the right word. Could they define why it's the right word? No. There's no way of explicitly articulating why you've chosen a word. It's come from somewhere, somewhere opaque that you don't really understand,

Speaker A: 00:32:25

and that's what GPT-3 is. Feels very similar, yeah. And it's the word shagtacular. Yeah, that's right. I think to put a kind of positive spin on this, I suppose, people might worry, oh no, journalism, speechwriters, they're going to get, no. See, I think the whole point is that GPT-3 and its ilk are going to be able to write all of the boring stuff in much the same way that machines have automated the boring aspects of industrial life. We're going to see a similar thing in, if most press releases produced by boring cosmetics companies are just going to be automatically produced, automatically ingested, and automatically put into the business pages for no one to read, great. Because that leaves more journalists to go out and break the next Watergate

Speaker B: 00:33:13

scandal. And then the NLP can read those articles and act on them. Jordan? I was just going to say,

Speaker C: 00:33:22

I came across a Guardian article, it was a column that was written by GPT-3, and it caused a lot of outcry online in the Guardian, but it was really good. And it was a really self-deprecating and self-reflective column about automation and robots writing articles. And it did make me think, yeah, I mean, even opinion columns and stuff, there's no reason why they couldn't really be written by GPT-3.

Speaker A: 00:33:54

And in fact, the more your opinion can be written by GPT-3, the more boring your thoughts are. So perhaps this could be a good test, is how surprising, how much entropy there is in your opinions is directly related to how reproducible they would be. You sounded quite a predictive text there when you said that would be a good text. Was it the opposite of that? I don't know. All right, look, we need to finish. I want to ask a question. I don't know if I've got a question

Speaker B: 00:34:23

or not. It doesn't really lend itself to a question, I have to say. Well, I'll tell you what, I'll

Speaker A: 00:34:30

predictive text a question. Yeah. Which application would you like me to use? Text messaging? Don't care, whatever. Okay, and then you guys can predictive text. I'll use Slack. Okay. Yeah, so are you doing anything today or tomorrow morning? And I will be there in the morning. And I will be there in the morning. Okay, so that's the question. Are you doing anything today or tomorrow morning? And I will be there in the morning and I will be there in the morning. So if you start answering with I will. Yeah. And then we can find out what the answers to my question are. So let's start with Chris. No more prawns, please. If we can avoid the prawns,

Speaker B: 00:35:16

that'd be great. I will be there in the morning. And I can confirm that you are not the same

Speaker A: 00:35:31

as you know. Correct. Jordan, what's going on in the morning? In the morning? In the

Speaker C: 00:35:45

morning, just to get a you. Yeah. The best way for me to get I probably best we stop there.

Speaker A: 00:36:02

And Fraser, what are we doing in the morning? I'll be in touch again when we arrive here at our place on Saturday. Awesome. I'm really looking forward to that. Well, I think that's a perfectly good place to end. I think it is. I think it is. Nice. Good idea, Nick. I like that. Okay. All right. We'll stop there. Thank you, as always, for listening to the Cognitive Engineering podcast. I'm Fraser McGruer, being here with Chris Wragg, Jordan Fermanis and Nick Hare of Aleph Insights. Until next time. Goodbye.