#93 Prof. MURRAY SHANAHAN – Consciousness, Embodiment, Language Models

Murray Shanahan is a professor of cognitive robotics at Imperial College London and a senior research scientist at DeepMind. He graduated from Imperial College with a first in computer science in 1984 and obtained his PhD from King’s College in Cambridge in 1988. He’s since worked in the fields of artificial intelligence, robotics and cognitive science. He’s published books such as In Bodyman and the Inner Life and the technological singularity. His book in bodyman and the Inner Life was a significant influence on the film X Machina for which he was a scientific advisor. Now, Professor Shanahan is a renowned researcher on sophisticated cognition and its implications for artificial intelligence. His work focuses on agents that are coupled to complex environments through sensory motor loops such as robots and animals. He’s also particularly interested in the relationship between cognition and consciousness and has developed a strong understanding of the biological brain and cognitive architectures more generally. In addition, Professor Shanahan is interested in the dynamics of the brain, including metastability, dynamical complexity and criticality as well as the application of this understanding to machine learning. He’s also fascinated by the concept of global workspace theory as proposed by Bernard Baas. We’ll be talking about that on the show today, which is based on a cognitive architecture comprising a set of parallel specialist processes and a global workspace. Professor Shanahan is committed to understanding the long-term implications of artificial intelligence, both its potential and its risks. His research has been published extensively and he’s a member of the External Advisory Board for the Cambridge Centre of the Study of Existential Risk and also on the editorial boards of Connection Science and Neuroscience of Consciousness. Conscious Exotica Professor Shanahan wrote an article called Conscious Exotica in 2016 where he invited us to explore the space of possible minds, a concept first proposed by philosopher Aaron Sloman in 1984. Now this space is comprised of all the different forms of minds which could exist from those of other animals such as chimpanzees to those of life forms that could have evolved elsewhere in the universe and indeed those of artificial intelligences. Now in order to describe the structure of this space, Shanahan proposes two dimensions, the capacity for consciousness and human likeness of the behaviour. According to Shanahan, the space of possible minds must include forms of consciousness that are so alien that we wouldn’t even recognise them. He rejects the dualistic idea that there’s an impenetrable realm of the subjective experience, remember we were talking about Nagel’s bat on the Charmage show, insisting instead for nothing is hidden metaphorically speaking, citing Wittgenstein actually. Now, Shanahan argues that while no artifacts exist today, which has anything even approaching human likened intelligence, the potential for variation in artificial intelligences far outstrips the potential for variation in naturally evolved intelligence. This means that the majority of the space of possible minds may be occupied by non-natural variants, such as the conscious exotica of which Shanahan speaks. Now ultimately Shanahan’s exploration of the space of possible minds invites us to consider the possibility for human-like minds, but also for those that are radically different and inscrutable. He concludes that, although we may never understand these alien forms of consciousness, we can still recognise them as part of the same reality as our own. So, Professor Shanahan has just dropped a brand new paper called Talking About Large Language Models, in which he discusses the capabilities and limitations of large language models. Now, in order to properly comprehend the capacities and boundaries of these models, we must first grasp the relationship between humans and these systems. Humans have evolved to survive in a common world and have cultivated a mutual understanding, reflected in their ability to converse about convictions and other mental states. Conversely, AI systems lack this shared comprehension, so attributing beliefs to them should be done circumspecally. Now, prompt engineering is something that we’ve all become very familiar with, we’ve discussed it a lot on this show recently, and it’s almost become a fact of the matter when it comes to these large language models. It involves exploiting prompt prefixes to adjust the language models to diverse tasks without needing any supplementary training, allowing for more effective communication between humans and machines. Nevertheless, lacking a more profound understanding of the system and its relationship to the external world, it’s difficult to be certain whether the arguments produced by a large language model are genuine reasoning or simply mimicry. Large language models can be integrated into a variety of embodied systems even, such as robots or virtual avatars. However, this doesn’t necessarily mean that these systems possess completely human-like language abilities. Even though the robot in the say-can system is physically embodied and interacts with the real world, its language is still learned and used in a dramatically different manner than humans. So in summary, although Professor Shanahan concludes that large language models are formidable and versatile, they’re fundamentally unlike humans and we must be wary of ascribing human-like characteristics to these systems. We must find a way to communicate the nature of these systems without resorting to simple terms. This may necessitate an extended period of interaction and experimentation with the technology, but it’s a fundamental step if we are to accurately portray the capabilities and limitations of large language models. So anyway, without any further delay, I give you Professor Murray Shanahan. Professor Shanahan, it’s an absolute honor to have you on MLST. Tell me a little bit about your background. My background. Well, I’ve been interested in artificial intelligence for as long as I can remember since I was a child, really, and I was very much drawn to it by science fiction, by science fiction movies and books. Then I studied computer science right from when I was a teenager. I got very much drawn into programming, was fascinated by programming. I really did my 10,000 hours of programming experience when I was quite young and I went on to do computer science at Imperial College London. That was my degree and then still fascinated by artificial intelligence. I went on to Cambridge and did my PhD in AI in Cambridge, very much in the symbolic school then. Then I had a long affiliation with Imperial College, did my post-doc there and still in symbolic AI. Then at some point I became a bit disillusioned with symbolic AI and I kind of segwayed into studying the brain, which was the obvious example of actual general intelligence that we have. I think I was a good 10 years on an excursion into neuroscience and computational neuroscience and that kind of thing. Then deep learning and deep reinforcement learning happened in the early 2010s and AI started to get interesting again and I got very much back into it that way. I was particularly impressed by DeepMind’s DQN, the system that learned to play Atari games from scratch and I thought that was a fantastic step forward and I really went back to my roots and back to AI at that point. I think we’ll talk about DQN when we speak about your article on consciousness. Having such a diverse set of experiences in adjacent fields, how have they influenced each other? Yeah. One thing I didn’t mention is that I’ve also had a long standing interest in philosophy and I very often think that by him is a weird kind of philosopher really and philosophical questions have had a great attraction for me. I think there’s a three way into relationship between artificial intelligence, neuroscience and the other cognitive sciences and philosophy and I think they all mutually inform each other really. Fantastic. So you wrote a book called In Bodyment and the Inner Life. What motivated you to write that book? Yeah. So at that point, so that book was published in 2010 and it was the culmination of a sort of long excursion into thinking about consciousness and about brains which took place after I had moved away from symbolic AI really. So I was thinking about the biological brain in the back of my mind. I’d always been fascinated by these philosophical questions about consciousness and then I went a bit kind of crazy and started thinking about these things seriously. It became kind of my day job to think about neuroscience and about consciousness. And around about that time, the science of consciousness was taking off as a serious academic discipline with proper experimental paradigms. So that was really fascinating. And I got to know Bernie Baas. Bernie Baas is the person who originated global workspace theory, global workspace theory being one of the leading contenders for a scientific theory of consciousness. And I was very drawn to global workspace theory and partly because it was a computational sort of theory. It drew very heavily on computer science and computer architectures. There was a computer architecture at the center of the theory. So this kind of collection of interests along with my philosophical interests all came together. I wanted to put them all into a book where I expressed my ideas about first of all from the philosophical side, very heavy influence of Vic and Stein, about how we address these problems at all. Then lots of global workspace theory and a certain kind of global workspace architecture, how that might be realized in the brain, drawing also on the work of Stanislaus DeHen who was working on what he called the global neuronal workspace idea. And putting all these things together into one big book. Amazing. Well, we’ll speak a lot about Vic and Stein when we speak about the language model paper and your consciousness paper. But two things that did trigger or prick my ears up, computationalism, which is quite interesting because some folks in the cognitive science arena are especially with the forees like examples to escape from computationalism. We did a show on Sells Chinese room argument the other day. He’s probably one of the most known people who do issue computationalism. So what do you think about that? Yeah. Well actually, so when I was talking about global workspace theory, I mentioned that it sort of comes out of a kind of computational architecture. But in fact, where I took it was very much moving away from that original presentation, which drew heavily on a kind of quite an old fashioned architectural perspective sort of boxes and how they communicate with each other and so on. And I was much more interested in taking it in a direction which is very much more connectionist and drawing much more heavily on the underlying biology and neuroscience, which in fact is also a direction that Bernie Barnes himself had moved him because the book that originally put forward his theory is from 1988. So that was the predominant way of thinking at the time was this very computational cognitive perspective. So by 2010 when my book was published, I was very much more interested in a kind of more connectionist perspective on things. And so that’s the way that it’s portrayed and in the book the theory fascinating. Because in the Serena, some people cite penrose and you know, or the need for hyper computation. Because people talk about the church during hypothesis and this idea that the universe could be made of information, which is quite interesting. But do you believe that the world that we live in could be computationally represented and computed? Well, I’m not sure that I have a belief on that particular one. So I mean, I mean, penrose’s ideas about consciousness will of course draw heavily on quantum mechanics and he thinks that quantum effects are important for consciousness. But I mean, that’s very much a minority, a tiny minority view within the people who study consciousness from a scientific standpoint. And so I don’t really subscribe to that point of view, I have to say. Well, I mean, coming at it from a slightly different angle, we spoke to Nome Chomsky recently and I’ve just done some content on Nagel’s bat. A couple of rationalists. There they are giving is about the subject of experience and the limits of our cognitive horizon and the inability really for us to reduce things into a comprehensible framework of understanding. So how would you bring that in? Yeah, well, gosh, I mean, yeah, we’ve launched right into some really big, difficult topics here, right? So in my book, embodiment in the inner life, which at the time I thought I’d really kind of like wrapped up the problem of consciousness. But one of the big sort of outstanding things for me in that one of the outstanding questions that I have not really answered I felt in that book is very much related to Nagel’s question about bats. What is it like to be a bat? So and it’s to do with the idea that there’s a sort of intuitive idea that maybe there can be you know, very exotic entities, very exotic creatures who are completely unlike us and yet somehow there’s some kind of consciousness there that we could barely grasp its nature and and this is a you know, it’s a sort of natural intuitive thought and especially when we look at other animals like bats and especially if we look at an animal that’s a bit different from us and we you know, we get hints that there’s that there’s some someone at home as it were and that there’s consciousness there we I think you know, we I’m sure all of us believe that cats and dogs you know and and many other animals are conscious and are capable of suffering and and you know, having awareness of the world that’s like our awareness and are aware of us and each other and so that’s it. I think that’s I mean, I take that as almost axiomatic. That’s just the way we treat those creatures. But then when we think about something like a bat and it’s very different from us so we you know, the natural thing thought is that maybe maybe what it’s like is very very different from what it’s like for us and it’s a natural thought to express. And and of course, you know, Nagel takes that thought to suggest that there are that there’s something that is inaccessible to us which is you know, what is it like to be a bat is something we can never know and this is a very un-victch and stiny and thought and I’m very much you know, I’m very attracted to to to Vick and Stine’s philosophy and but but it’s also a very natural thought that you know, so there is a very un-victch and stiny and thought because Vick and Stine says for example, you know, nothing nothing is hidden. So he’s very you know, and the whole private language remarks are all about sort of saying, well this this this intuition that we have that there’s this private realm of experience is actually just we’re just it’s just it’s just a philosophical trick of the mind to think that this sort of peculiar metaphysical realm exists of inaccessible subjective experience in in in others and and that’s the that’s his whole thrust of his philosophy or of that you know, that aspect of it is to try and undermine that. So these two things are intention right? So there’s this natural thought that that that’s you know, there must be like something to be a bat, but what is it like and how could we ever know and then there’s the Vick and Stine and thought which is actually very difficult to kind of really embrace but but it’s that there’s a sense in which nothing is really metaphysically hidden it’s only hidden could be hidden empirically because maybe we don’t know enough, I mean we haven’t hung around with bats often in an awful maybe we haven’t examined their brains or maybe but that’s all empirical right? So there’s nothing metaphysically hidden whereas Nagel’s point is that there’s something that’s deeply profoundly philosophically metaphysically hidden which is the subjective. Now we can extend that to carry on. So I’m rambling now. So we can now we can extend that thought about bats now you know, especially from the perspective of the sort of thing that I’m interested in to why not just bats but what about the whole space of possible minds to use air and slowments, very evocative phrase? What about you know extraterrestrials who are going to be you know whom surely there are there is extraterrestrial intelligence out there it’s going to be very very very different to us. So and then what about the things that we build? Maybe we can build things you know and artificial intelligence of the future maybe you know we can build something that is also conscious it’s kind of things depicted in science fiction all the time. Science fiction is often depicted as very human like but there’s no reason why it should be human like it all and so we can imagine these very very exotic entities and then the question is even bigger you know there could be something that we we want to even be able to recognize that there was even the possibility of consciousness but maybe it’s buried there inside this complex thing somehow. So that’s the that’s the kind of question that that fascinated me and I wrote this paper called Contrace Exotica which is all about trying to trying to make that thick and stinging perspective encompass this possibility as well. Yeah and and maybe we should talk about that before the language paper just because it’s it’s what we’re talking about. But there’s a few things you said there which are really interesting so you know when when Chomsky talks about ghosts and the machine and he goes back to Galileo and and Descartes and actually it was Descartes who you know introduced this kind of mind body dualism and you know which was kind of a move away from the previous desire to have a mechanistic understanding of the world that we live in. Humans want to understand and actually so many things in the world alludes our understanding and then that brings us on to David Choms coined that the hard problem of consciousness which I suppose is an extension of the mind body problem and it’s as you were saying this little bit extra right so we think about and I agree with Choms that intelligence and consciousness are likely entangled or would would co-occur together but he always said that there’s function dynamics and behavior and then there’s that little subjective thing on the top and for Choms consciousness it’s almost like what’s the cash value of it he just thinks it’s just something on top it’s not really requisite for anything else and I believe it might be requisite for intentionality and and agency as as so did but what should I take. Well it’s interesting because the the whole way that you put that and the whole way that people often talk about this thing is you speak about consciousness like there’s this thing which you know there’s this singular thing which maybe it’s needed maybe it isn’t maybe it’s this maybe it’s owner but I think that that whole way of talking is is which is natural for us in many everyday situations but when it comes to this kind of conversation I think that whole way of talking is maybe not quite right because we’re thinking of consciousness as this you know we’re reifying it turning into this thing whereas I think maybe at that point we have to take a step back and we have to say well when we talk about when we use that word conscious or consciousness so we use it in all kinds of different ways in different contexts and so when we talk about you know we might talk about it in the context of an animal we might say well this the animal you know this dog is aware of its environment so you know this dog can see the the bowl in front of it it can see me it can see the door it can see the trees it can see the squirrel you know and and it can smell more like you’d smell all of these things as well so so so so we use consciousness you know we talk about consciousness in that that that sense and we also talk about our self-consciousness you know we talk about the fact that we’re you know we’re aware of our own thoughts and and we we talk about our inner life and and we use consciousness to to encompass that as well we we often use consciousness in the context in science scientifically of a distinction between conscious and unconscious processes and that’s a very interesting distinction because when we are consciously aware of a stimulus as humans then a whole lot of things come together our our we’re able to kind of like deal with novelty better we’re able to report it we’re able to remember things better so so whereas when we perhaps are unconsciously or there’s a kind of unconscious processing of a stimulus then we still can respond to it behaviorally but and it can have queuing effects and so on but it doesn’t have all those other things so this and and that’s kind of there’s a kind of integrative function for consciousness there and then on top of all of that there is the capacity for suffering and joy that comes with so often there’s there’s valence to consciousness you know there’s there’s there’s so that’s another thing so all of these things they come as a package in humans but when we speak about edge cases then all then then the these things come apart and we need to speak about them separately I think fascinating when we know there are two kind of minor digressions there I mean that you were talking about these planes of consciousness which is also very interesting and maybe we could get into the integrated information theory or the global workspace theory just just for the for the audience just to give them some context yeah sure or John we say a few words about oh please yeah yeah yeah okay yeah so there’s so there are a number of kind of candidates for for a scientific theory of consciousness and you just mentioned two of the leading ones which are global workspace theory and integrated information theory and so global workspace theory so that’s that’s Bernie Barz’s was originated by Bernie Barz and has been developed by Stanislaus Duhenn and colleagues so the idea there is it’s it does rest on this sort of architectural idea which is that which is that we think of of the brain the biological brain as comprising you know a very large number of parallel processes this is kind of a natural way to think of the brain a large number of parallel processes and and the global workspace theory posits a particular way in which these these processes interact and communicate with each other and that is via this global workspace and the idea there is that is that there are sort of two modes of processing that that go on so in one mode of processing the these parallel processes just do their their own thing independently and in the other mode of processing they are working via this global workspace theory so the idea is that they you might think of them as as you know depositing messages if you like in this global workspace which are then broadcast out to all of the other processes so it’s so there’s this kind of but I think thinking of it in terms of messages is not quite the right way of thinking of it is better to think in terms of kind of signaling and information and so on but that’s a natural way to think of it but so these so in in that mode these processes are sort of disseminating their influence to all the other processes and that’s the global kind of broadcast aspect of it that’s when consciousness what that’s when information processing is conscious according to global workspace theory as opposed to when it’s all just local and the processes are doing their own thing that’s that’s not that that processing is not conscious so there’s a distance so it’s about teasing out this distinction between conscious information processing and unconscious information processing now all of those terms by the way are deeply philosophically problematic into going you know you have to sort of do it properly you have to kind of unpack them all in very carefully and that’s what my book tries tries tries to do but so essentially it’s about it so the the essential idea though is to do with broadcast and dissemination of information throughout the brain and going from like local processes and having global influence and that’s what consciousness is all about according to global workspace theory okay so integrated information theory so I think so integrated information theory which is Giulio Tannoni’s theory which Giulio Tannoni thinks there’s is kind of incompatible in some ways with with global workspace theory but I don’t think that’s that’s true I think I think that there’s a lot of synergy between the two theories in fact but that’s because they so they come with so to integrate the information theory has sort of two aspects to it so so according to Giulio Tannoni he really is trying to pin down a property which is almost like a physical property which is identical with consciousness so you can actually speak about the amount of consciousness in any system that you that you look out phi he grew this it’s phi so the phi is a number how is actually a number of how much consciousness is present in the system like like part of your brain or your whole brain or you as a person or a flock of bats or whatever so you can or or or toaster you know so you can give a number to how much consciousness there is there according to his theory and it’s a mathematical theory based on Shannon’s information theory and it’s but it’s all about trying to see how much information is processed by the individual parts of the system versus how much information is processed by all the parts put together and it’s and it’s to do with how much the second thing you know exceeds the first thing and in a sense and that is how much consciousness there is there and and in a way it it actually has some synergies if you as long as you don’t think that it’s necessarily measuring you know this property of the of the universe which you can put a number on but it has some synergies with global workspace theory because they’re both distinguishing between global holistic things versus local things and the and the consciousness is in the kind of global holistic processing versus the local you know a local processing in both those theories so there’s a kind of you know there’s some intrusions that they have in common I think interesting and it also reminds me a little bit a little bit about what Charm is speaking about so he he he thinks that it strongly emerges from certain types of information processing and the processing must represent causal structures as well so it can’t it’s it’s not an appeal to pan-psychism per se and although with with all of the things that you’ve just spoken about and what do they work in another universe and I guess what I’m saying is is it just the the physical and the information processing or in a different universe might it not emerge in the same way yeah we’re suppose what you mean by a different universe I guess what do you mean by a different universe well if the laws of nature were different yeah okay so if the laws of physics were yeah well we’re different well I guess my I guess I I dislike isms I’m I’m I’m an anti-ismist or rather I say I’m not an ismist but if I were to I but I do sort of subscribe broadly to functionalism I suppose so I guess yeah I guess I what do I mean by that I mean what I mean is I mean I really dislike saying that I subscribe to these to these isms so what I really mean by that is that is that I imagine that a system that is organized in a particular way functionally in terms of its information processing and if that system is embodied in the broadest sense and you know and beats lots of other pre-requisites then it’s likely to behave in a way where I’m going to naturally use the word conscious to describe it perhaps and where I’m going to treat it like a fellow conscious creature so so so it’s so you know ultimately it’s I think it’s about the kind of organization you need to give rise to the behavior you need to talk about thing the thing in a certain way my question to that because I I posed this question to charmer’s last week he’s also a functionalist and and I agree with the degree of functionalism describing intelligence but less so with with consciousness you know there’s not a cheering test for consciousness for example but the the thing is with functionalism I we’re at risk of doing what you said people do have large language models which is anthropomorphizing them because these functions are intelligible to us and then our conception of intelligence becomes somewhat observer relative yes do I mean what I observe a relative so as a you understand these functions so it’s conscious to you but not to someone up well so so so in all of these cases I mean I think it’s about the the words that we use in our language to talk about the things so the so so if the someone else is someone just like us right then then we have to and if we want to use the words in different ways so so the the large language models are a great case in point right so so suddenly we’re arriving at a point where somebody can describe something as conscious and others can say that’s rubbish you know it’s not that’s not true at all and so we so we’ve we’ve arrived at a point where these philosophically problematic words which which we use in in ordinary life quite quite harmlessly and we all you know we all are in agreement about how we use the word like somebody says oh you know Fred has drank so much last night he passed out he was completely unconscious you know and we and or if he needs to just says yes the you know the patient is now unconscious they can’t feel feel pain or if you say oh you know I I just wasn’t aware I didn’t see the the cyclist you know that’s why I hit them you know I’m really it’s tragic but I just didn’t see them and then and we so you know so you’re saying I wasn’t aware of it so it didn’t influence my action so they were using the terms in ways that we all understand but now we’re getting to a point where suddenly these words or these concepts are being used you know we we don’t have an a way we don’t have agreement about how to use these words right because it’s there are these exotic edge cases yes so then the question I think that you you’re getting it is you know is is there a fact of the matter there right and so I’m very tempted to say the first thing I’m tempted to say is that I don’t think that perhaps is a fact of the matter or certainly I don’t I don’t want to I don’t want to speak as if there is a fact of the matter but rather I think we need to arrive at a new consensus about how we use these words so that might mean that we extend the words we break them apart like I was suggesting earlier maybe we need to separate out awareness of the world from self awareness from integration cognitive integration from the capacity for suffering because suddenly we have things that where that they don’t all come as a package and when we need to kind of be a bit more nuanced in the way that we use these words we use to use them in new ways but then there’s a kind of transition period because we don’t you know we’re all arguing about how to use these words on the sun because we’ve got weird edge cases so there’s going to be a time when it’ll take a time for language to settle back down again so there’s a kind of you know you there’s a kind of observer relativiveness to this for a bit if you like but then but then there’s a kind of consensus needs to emerge right but so many things to explore them in I mean I’m I would love it if this platonic idea of of concepts were possible and I think what platonic because what we’re talking about here is reductionism and the I mean the parable of the blind men and the elephant comes in quite nicely so as Chomsky said complex phenomenon beyond our cognitive horizon and as much as we don’t want to we use functions derived from behavior to have some common understanding of this thing but I wasn’t being reductionist was I do you think I was being reductionist what well well no so you said the language game converges and and in suncases we will arrive on on a common definition but I think you can bring in Hofstatt or as well well not a common definition but a common usage right so we’ll come so we’ll come to use the words you know in a and with agreement right so that’s what I and the reason why I mean I would and I worried the reason I would balker using the word reductionist is because and that’s why I’m a bit resistant to functionalism as well that any kind of ism is because I just think that that may be the way things are organized when you take them apart so you know brains right news when you examine them on the inside like animal brains you might look at how an octopus is brain works and that might inform whether you think that it suffers can experiences pain or not or we might break apart you know an AI of the system of the future right and you know and we make about breaking apart and we may look at its functional organization and that all is just is grist to the mill of how our language might change right so I’m not I’m not subscribing to the fact that consciousness is this or this is that it with some big metaphysical capital letters on the is right that’s really important and so so the so the so the organization the functional organization of these other things which when we study it and look at it is all just part becomes part of the conversation that eventually is going to help us to settle on maybe new ways of talking about these things I think we agree with it with each other I think the difference is so with the power of the blind men and the elephant all of the men around the elephant saw something which was part of the truth yeah and I think that’s what we’re describing with the function so we can all agree on what perception means or what some action means yeah yeah the thing is there will be many other functions that will represent a different slice of that cognitive phenomenon yeah I agree and I think that’s very much true with consciousness actually because because lots of people coming with kind of like new ideas and new theories I mean you know anil Seth for example I don’t have you had anil on your on your not yeah being you right yeah so anil’s written this great book called being you yeah and and and anil brings in a whole kind of you know new set of ideas he’s particularly interested in you know the sort of top down effects of on on perception top down effects on perception so then he brings in this kind of top down influence and perception as a big part of things and then there’s Graziano has written this book using this about his attention schema theory of consciousness and and that’s and you know there’s a whole set of interesting ideas there and I think you’re absolutely right I think there’s I think there’s aspects of all of these things all feed into you know all feed into the way you know brains and animals work and all of them feed into the you know why they behave the way they do and why we use those words when we use those words you know conscious and aware and so fascinating we’ll get to your article in a second but as someone who has such a diverse and interesting background who is allowed to ask these philosophical questions so it reminds me and Thomas Meckensew is talking about the arguments between neuroscientists and philosophers about freedom of the will yeah and who who gets to decide huh yeah well what a great question you know I mean so so why should I have any right to speak about any of these things dogs I have no formal training in philosophy so so so who gets to who gets to this well who gets to to I guess there are two things right there I guess I guess there’s there’s in in that kind of discussion between the neuroscientists and the philosophers so they’re you know they’re not talking about you know the everyday conversation that we’re all having as as as humanity or as English speakers or as Chinese speakers about how we use these these these words so there it’s a much more kind of confined to the to different two different schools over at or disciplines within academia so there I mean I do think that the people who work in AI and in neuroscience probably at least should be a bit familiar with with the philosophical debates and you know you mentioned Descartes earlier on and you know you’re familiar with with just that that basic kind of you know it’s sort of stuff that which is like philosophy 101 which people should at least be aware of Descartes arguments and then charmers and the different kind of perspectives on those sorts of things before they pitch in you know at least I mean you wouldn’t pitch in to neuroscience just by making some up some stuff about brains if you hadn’t read you know the an introduction to neuroscience and so so I think that the scientists need to you know you know they need to kind of have a a pass to enter the conversation which is to have to have got through philosophy 101 yeah it’s so true we we have the same thing with the with the ethics folks actually because because we we have a lot of them fields of expertise and engineers should learn more about ethics yeah absolutely but when they do have an opinion about ethics quite quite often it’s it’s um you know it can sometimes be a bit naive and and and and you know at least you should be familiar with the kind of but but that’s an interesting and a difficult error in itself because of course you know you as a scientist it’s important that you take responsibility as a scientist and and that you take you know some ethical responsibility but at the same time you know you’ve only got so much time to become an expert so so it’s difficult to at the same time take ethical responsibility um and yet you know even though perhaps you haven’t got the time to kind of read you know read up and become an expert on the relevant ethics so I mean perhaps everybody again should you know get to the entry level you know ethics 101 and indeed many many courses teach you know ethics these days many kind of science and computer science it’s part of you know of of any degree these days so that’s a good step I think yeah there’s an interesting analogy between the functionalism that we were speaking about in consciousness I mean even in our own research domain we we have the function of ethics and we have linguists and we have all sorts of different people yeah and and and that is the blind men in the elephant and you know I tend to believe that even though the the views from these diverse folks are inconsistent diversity is very important oh incredibly important intellectual diversity is you know every kind of diversity is important and intellectual diversity is immensely important and having these conversations is interdisciplinary conversations is absolutely you know essential so at least if people are talking to each other that’s a really really positive thing I think fantastic now we invited charmers on our podcast after Ilya Sudskevers tweet and by the way charmers took a very functionalist approach to talking about consciousness but I guess after that tweet everyone in the community started thinking about and talking about consciousness so maybe let’s just start from that tweet how did you find it sure yeah okay so the tweet was so so Ilya Sudskevers said it may be that today’s large language models are slightly conscious and and then that I repart replied tweeted back in the same sense that it that may be a large field of wheat is slightly pasta and and that in fact was it was actually I mean I’ve got a fair number of Twitter followers and that was the most engaged tweet I’ve ever sent out and you know and you know it got celebrity likes hand-off ride retweeted it and you know all the owners might kind of comment and so but that does kind of summarize sort of what I think about about what he said at that point but then but then I after tweeting my my flippant response I then I’m violating all my own Twitter rules in just sending back a flippant response because I generally don’t do that I would rather kind of you know be professional engage professionally and so I thought it was very important to follow that on with that you know with a little explanation of why you know why I thought that it wasn’t really appropriate to speak about today’s large language models in those terms yeah and for me the number one thing is to do with embodiment so so as I see it embodiment is a kind of prerequisite for for us being able to use that that word use words like consciousness and so on you know in the way that we do in the normal every day way of talking so so you know it’s only in the presence of something that that inhabits our world and by inhabits I don’t mean just has a physical you know like a computer is obviously a physical thing in our world but inhabits our world means that you know walks around in a herald swims or jumps or flies or whatever but is is is is is inhabits the same world as us and interacts with it and and and you know and interacts with the objects in it and with other with other creatures like ourselves so so that to my mind that is that’s the that so so so I so in that paper conscious exotic I think I use this phrase trench channeling victgenstein that that only in the context of something that that exhibits purposeful behavior do we speak of consciousness and the way that that’s phrase that there is kind of you know so so try to channel victgenstein’s style of of of saying things because you notice that he’s saying that it’s only he’s making what he’s saying is actually he’s talking about the way we use the word so he’s not making a metaphysical claim is essential he’s saying that this is just this is these are the circumstances under which we use this word so we use this word in the context of fellow creatures basically and so so that’s kind of the starting point so a large and of course we of course we talked to people on the telephone and over the internet and so on and we don’t you know we may not we you know we can’t see them or anything so we but but but we still we know that there is you know or we assume we’ve always been able to assume up to this point that there is a fellow creature at the other end and that’s the kind of grounding for kind of thinking that way and using that using that word I mean that is absent in large language models so large language models do not inhabit the world that we do and now I mean we have to caveat that because of course it’s possible to embed a large language model in a in a in a in a we always do you embed it in a larger system might be very simple and betting it might be just a chatbot or it might be much more complicated like it might be be part of a system that enables a robot to kind of move around and interact with the world and take instructions and and so on so there was a great some great work from Google with their palm say can robot for example where there’s this embedded large language model so so so there you’re kind of moving in a in a direction where maybe where these where these words you know the prerequisites you know for for well actually I want to be careful what I say here sorry because it’s so easy to say something that’s going to can be misinterpreted right but but we can’t imagine that that we can’t imagine that requirement being met for for for for not not it doesn’t mean it wouldn’t be a sufficient condition for using those words but at least it wouldn’t you’d meet the necessary conditions right but the large language models by themselves do not meet even they’re not even candidates I yes I agree and we there’s so many things we can do here because we can we can talk about embodiment in general I mean as I understand it Rodney Brooks kind of started the phenomenon of thinking about a representationalist view of artificial intelligence or in rejecting rejecting a representing a rejecting yes so so Rodney Brooks thought that we should use the world as it’s sign best representation which is absolutely fascinating yeah and and then you you maybe you might be thinking about the embodiment view in a similar style of a Wittgenstein so it’s a matter of complexity and it’s also a matter of encapsulation which is fascinating but but also just to quote your paper you said although the language model component of say can provides natural language descriptions of low level actions it doesn’t take into account what the environment actually affords the robots and there’s this whole affordance thing as well so so I mean how do you think about embodiment so so the way I see it is that is that the you know the one example we have as of you know the end of 2022 of something that we really can describe as in as as as intelligence as generally intelligent is is the biological brain biological brains of humans but also of other animals and the biological brain you know at its very it’s very kind of nature is it’s there to help a creature to move around in the world to move right it’s there to move help to guide a creature and help it move in order to help it survive and reproduce that’s what brains are for so that’s what that from an evolutionary point of view that’s that they they they developed in order to help a creature to move and they are and so they they and they are you know they’re the bit that comes between there’s the sensory input and the motor output and in surprise you can clearly divide these things which maybe you can’t but I mean so and so that’s that’s that’s their purpose is to intervene in the sensory motor loop in a way that benefits the organism and everything else is on built on top of that so so so the capacity to to recognize objects in our environments and categorize them and the the ability to kind of manipulate objects in the environment pick them up and so on and all of that is there you know initially to help the the the organism to survive and and and and you know and that’s what um brains brains are there for and then then when it comes to like you know the ability to work out how the world works and to to do things like figure out how to gain access to some item of food that’s difficult to get hold of then all kinds of cognitive capabilities might be required to understand how you get inside a you know a shell or something to get it the fruit inside it or something like that complex cognitive abilities that’s so that and then you know evolutionary evolution has developed a lot of more and more complex cognitive cognition until we get to language and you know we need to interact with each other because that that’s all very much a part of it the social social side of it and the language is part of that so as I see it it’s all built on top of this fundamental fact of of the embodiment of of the animal and the organism so that’s in the biological case so that’s the sort of our starting point yeah so um and so that seems to me to be the the most natural way to to understand the very nature of intelligence good I think I didn’t frame it very well I mean Melanie Mitchell recently had a paper out talking about the four misconceptions and AI and one of them of course citing Drew McDermott was the wishful mnemonics and the anthropomorphisation which which she basically spoke about but but her fourth one was about embodiment and she spoke about this in her book as well and she said that one of the misconceptions of AI is that people have this notion of a pure intelligence you know something which works in isolation and environment and you’re talking about social embeddedness and embodiment and I guess my point with the complexity argument is I’m saying that the brain itself doesn’t actually do everything it kind of works as part of a bigger system oh I see what you mean yes okay yeah yeah so there’s um uh so of course there’s I mean I noticed in one of your previous interviews with Andrew Lampin and you mentioned the three E’s framework or four E’s four E’s these days um uh and as of course that’s very much part of it is the is the idea that you know there’s the extended mind we use the environment uh you know as as a kind of memory for example for we we deposit things in the environment writing you know as an example and so on um and then there’s um people talk about morphological competition I’m sure you’re familiar with that so no so what what’s that well so that’s the idea that the very shape of our bodies you know is is is is you know could so so so sometimes you know the aspects of intelligence are actually outsourced into the physical shape of of your bodies so where you might think about designing a robot where you where where you put a lot of work into the control aspect of it so that it’s so that it kind of kind of walk in this very carefully engineered ways that it’s always permanently stable or alternatively you can make a body that is naturally sort of stable or maybe naturally unstable and what you do is you kind of rely on the combination of the physics of it constantly falling with uh with a control system that constantly restores balance so that so you know that so it so that’s that’s what I mean and this is very much a Brooks type perspective and and people picked up on Brooks’s ideas and extended them in this sort of way so I think that’s I think that’s a very natural uh where thinking but in in in a way that this gets to the to the complexity argument because I guess what I’m saying is that um life is much more brittle than anyone realises you were just pointed to some sources of writtenness than most people never would have thought of hmm which is which is fascinating so I think there’s a very important relationship between embodiment and language uh and they also brings us back to victim style as well so um uh so for us as humans language is inherently an embodied phenomenon it’s it’s it’s it’s something that is uh uh it’s a social practice it’s something that take that that it’s a phenomenon that um uh occurs in the context of other language users who inhabit the same world as we do and where we have kind of like joint activities we’re triangulating on the same world and we’re we’re acting on the same world together and that’s the that’s what we’re talking about when we use language so there’s this uh so that that’s a inherently convict and stinging view of language I mean I think it’s deeply profoundly correct to have view of of language that’s that’s what that’s what language is there for us is so that we can talk about the same things together so that we can our collective activity is is you know is is is organized um to some extent thanks to language so that’s so I think that’s a really important perspective uh or language is thick and stiny and perspective and embodiment is at the heart of it embodiment and inhabiting the same world as our other language users um and you know that’s the way we learn language we learn language um by being around other language users like our parents and carers and and and and and peers uh and and that’s again a very important aspect of of the of the nature of human language now large language models um they learn language in a very different way indeed they do not inhabit the same world as us they do not kind of sense the world in the way that we do they don’t learn language from uh peer from other language users from their peers in the way that we do um but rather they’re you know we’ll be know how large language models work there’s trained on a very very large corpus of textual of textual data so an enormous corpus of textual data so bigger than any human is likely to encounter you know you know by the time they become a proficient language user at a young age um and what they’re trained to do is is um is not to kind of engage in activities with other language users but to predict what the next you know what the next token is going to be which is a very very different sort of thing so they’re very very different sorts of things and the and the role of embodiment is really really important in this different saying yes um absolutely when i spoke with Andrew Lamponen um he’s really really interested in the grounding problems i mean would you mind just speaking about that a little bit before we go into your paper yeah absolutely yeah yeah um so of course this goes goes back to a great paper by Stephen Harned back in I think 1999 or 1988 so the one and only yeah the one and only uh on the the symbol grounding problem it was called and uh and and you know he does um argue broadly that um uh that symbols in in in AI systems um the kinds of AI systems he was thinking about at the time were kind of you know sort of expert systems say or something like that and the symbols there are not grounded they’re provided by the human programmers and they’re just sort of typed in whereas for us for us were the words we use those symbols are are grounded in uh in our activity in the world so that when we use the words dog um that’s because we’ve seen dogs and we’ve talked about dogs with other people who’ve also seen dogs and we’ve seen dogs in lots of different circumstances and we’ve also seen cats and and uh and dog bowls and bones and many other things that all kind of contextualize that but all of that that that is kind of grounded in the real world in a now percept and in our perception of the things in question so that so that’s this so that’s what sort of is meant by grounding all that at least that’s the original meaning of the word grounding from Stephen Harlet’s paper yeah and I think that’s a really really important um concept because uh because uh you know in an important sense large language models the symbols that are used in large language models are not really grounded in that kind of way now this you know I should be absolutely clear that large language models are immensely powerful and immensely useful and and and so that you know so but it’s interesting that to what to what extent the lack of grounding here that we have in today’s large language models you know might uh affect uh how good they are so um so they you so they are prone to kind of you know hallucinating and and and and and confabulating and uh and if you look at multi-modal language models that that maybe we’ll talk about an image that you present to them uh then you know they you can have a very interesting conversation but sometimes they’ll go off piece and start talking about things that are not in the image at all and as if they are and um uh that’s sort of because due to a kind of like I was a lack of grounding this so that so the words are not kind of grounded in the images in in in quite the way that we would like so that’s it’s an important topic of research I think yes indeed and although some people do believe there’s this magical word called emergence and possibly some emergent symbol grounding might be possible maybe maybe let me just put that to bed before we introduce your yeah sure well uh well I mean emergence is is I think is uh is a really important concept and I yeah uh and I think uh you know we do see a tremendous amount of of uh very powerful emergence I think in today’s large large language models so so so uh you know even though that they’re so they’re basically trained to do next word prediction or I mean I’m just I have to be clear I suppose I should have made this maybe a bit clearer in the paper but of course it’s not always next word prediction because the different models learn to actually uh you know predict what’s in the middle of a of a sequence rather than kind of but generally you know they’re interested in in in in in in let’s take the next word prediction cases canonical so yeah so so so they’re so they’re trained to just to do next word prediction now the astonishing thing is as I think GPT-3 showed us is that is that just that capability if it’s sufficiently powerful can be used to do all sorts of extraordinary things because if you provide you know the prompt that describes you know describe some kind of complicated thing you know situation like uh you know I I need to um uh tile my floor and my floor is shaped like an L and it’s 20 meters long and three meters well you know you start to describe this thing you know and each each tile is is is 20 centimeters square how many tiles will I need and and some large language model will come back and tell you you need 426 tiles whatever and it’s correct right well this is astonishing because it was only trained trained to do next word prediction and so there’s a kind of emergent capability there now there’s a sense of course in which it still is just doing next word prediction because in the vast and immensely complex distribution of tokens in human text that’s that’s that’s out there then the correct answer is actually the thing that’s most likely to come up and that’s but it’s got to discover a mechanism for producing that right and so that’s where the emergence comes in and it I think that the you know these capacities are astonishing the fact that they that it can discover mechanisms you know uh emergently that will do that sort of thing yes and maybe I shouldn’t have used the word magic whether with emergence I’m a huge fan of emergence and as you say the the decoders train to predict the next token or the de-noising auto encoders to to um let’s say fill in the gaps in in the middle and I guess there are different ways of thinking about emergence so there’s weak emergence which might be thought as um computational irreducibility or a surprising macroscopic change or strong emergence when it’s not directly deducible from truths and the lower level to make you know a lot more talk about the circumstances of it yeah exactly but I guess my point is that remarkably it’s trained on something quite trivial so all of this is about convention right all of this is about what’s what what is the what is a good way to use words right so I don’t so I don’t think you know I’m not making metaphysical claims about about about these things so it’s all about what you know when is it appropriate to use words uh to use certain words and and because when it when this becomes problematic is when there are philosophically difficult words like beliefs and things and so on um now when it comes to reasoning so so I do think that we I do think it’s not unreasonable to to use that term to describe what some of the these models do today and that’s partly because of the content neutrality of of of of reasoning so so so so a lot of the argument or a lot of the discussion in the paper comes back to this sort of whole embodiment thing really um and and I’m I’m I’m saying well you know in in the kind of like ordinary way we use the word believes well it’s gets it gets complicated because we do use the word believes uh in this intentional stance way to to talk about ordinary everyday things we say oh my my you know my my car clock thinks that it’s a British summertime you know you know and because we and then but then you’d say then you then you somebody says to you what you mean your your car clock and think you say you know obviously I didn’t mean that it can think it’s just a turn of phrase you know but when we when we get to these large language models and we start to use the words like thinks and and believes and so on because they’re so powerful you start to get ambiguous and you’re and you know and you know when and some people say well actually I really didn’t mean that it can think whether it believes so I’m so I’m I’m interested in this when things get difficult in this respect and um could could you tease apart though what was so you resist anthropomorphic language in terms of belief knowledge understanding self or even consciousness yeah but less so with reasoning and I my intuition is that reasoning rather depends on those things that I just said before well I um so I think it doesn’t because um but but this is but perhaps this is just uh maybe enough kind of formal logic sense because because reason it because logic is content neutral so if I tell you that every could you just explain what you mean by that okay yeah so um so Lewis Carroll has all these wonderful kind of like nonsense syllogisms right where he where um uh you know he says oh if all elephants um like custard and you know Jonathan is an elephant you know Jonathan likes custard and you know all kinds of things like that and it’s all sort of nonsense and he has this big complex complicated ones similarly I could tell you that uh all all uh sprung fourths are clingy and uh and Juliet is a sprung fourth therefore Juliet is uh splinging right and we I’ve no idea what any of those things mean but the the but it’s because it because it for the pure form of the reasoning you don’t have to know what they mean it’s just about the logic so um so in that sense you know um uh it just in the way that a theorem prover can do logic then so can a large language model do logic so in that sense I think large it is reasonable to use the word reasoning in that logical sense in the context of large language models I don’t think that’s a problem of course we may think that they do it badly or they do it well or if that’s a whole other thing right but but at least the word is potentially applicable right yes now belief I think you know I think at the moment is a is a is a different kettle of fish because to really uh have a hold of belief it’s it’s not content neutral right so if you if I believe to use the example in my in my paper if I believe that uh barundias to the south of of of rowandah well whether that is the case or not it does depend upon facts that are out there in the world and then to to really have a belief at least you’ve got to be able to uh somehow try and kind of justify those facts or at least and you’ve got to be at least built in such a way that you can you know interact with the external world and and do that sort of thing right and verify that something is true or false or do an experiment or you know or or ask someone or you know you’ve got to go outside yourself right we go outside of ourselves and and uh in order to establish whether something a belief is true or not and so you know you’ve got at least be capable of doing that whereas large but language model the bare bones large language model is not capable of doing that at all right now you can embed it in a larger system this is a really important distinction that I try and make over and again in the paper I talk about the bare bones large language model so you can take the so so whenever a large language model is used it’s not the bare bones large language model which just does sequence prediction but it’s embedded in a larger system when we embed it in a larger system well that larger system maybe could consult Wikipedia maybe it could be part of a robot that goes and investigates the world so that’s a whole other thing but then you have to look at each case in point and and ask yourself whether it’s a whether you know whether we really want to use that word in it in in anger you know as in in its full sense rather than just in the intentional stance sense of a kind of figure of speech so and so in the case of of like chat bots for example um today’s chat bots not reappropriate I would say we’re not using the word in the way that we in the full blown sense that we use it when we talk about each other fascinating okay well let’s get on to intentional stance so you said that it’s a useful way of thinking about artificial intelligence allowing us to view computer programs as intelligent agents even though they may like the same kind of understanding as a human and you cited the case of Bob and bot the the word no was used differently in the two cases and the word of Bob it was used in the traditional sense for bot it was used in a metaphorical sense so it kind of like it’s distinguishing what it means to know you know for humans and and for machine so I think it’s it’s useful to think about something like Wikipedia so so we might ask the question does Wikipedia know that Argentina has won the 2022 World Cup and just immediately after the event you know it probably doesn’t it’s not recorded in Wikipedia and somebody might say oh Wikipedia doesn’t know yet that the Argentina of one and so when we use the word like that you know nobody’s gonna kind of say to them say to somebody who uses that word hey you know I don’t think you should use the word nose there or you know that was you know you should be a bit more sort of sensible I mean it’s it’s fine to kind of use I think these kinds of words in this ordinary every day sense we do that all the time and that sort of particularly particularly in the case of computers that’s adopting what Dan Dennic calls the intentional stance so we’re interpreting something as as as as having beliefs desires and intentions because it’s a kind of convenient shorthand and especially if you’ve got something that’s a bit more complicated like say your car sat nav or something or you’re you know you’re sat nav on your on your phone then it sort of makes makes sense to use those words it’s a is a convenient shorthand and it helps us to kind of talk about them right and without getting overly complicated without knowing the underlying mechanisms but there’s an important sense that if we don’t mean it literally so you know in the case of Wikipedia you can’t you couldn’t go up to Wikipedia and pat it on the shoulder and say hey Argentina of one and there’s no way you know right I want to be a you know in yeah and and and all the things that that go with us as humans actually knowing things and it’s just a turn of phrase now things get sort of interesting with large language models and with large language model based systems and the kinds of things that we’re starting to see in the world because we’re starting to get into this kind of blurry territory where where we’re blurring between the intentional stance and and you know meaning the meaning it literally and this is where we need to be really really kind of careful I think so at what point does do things shade over into where it’s legitimate to use that word you know literally in in the context of something that we’ve built you know I don’t think we’re at that point yet and we need to be very careful about about using the word as if we were using it literally you know that’s the sort of anthropomorphization because the problem is that we can then impute capacities to the thing and and or even you know empathy say that that that just isn’t there yes and I suppose we could tease apart and knowledge so it justified true belief from nose because nose that it brings all this baggage of intentionality and agency and anthropomorphization but you had chomp ski you’ve had chomp ski on I can tell you a story about that I mean the recording messed up so when we were interviewing him we were only getting bits and pieces and we had to deep fake him we had to we had to regenerate the interview oh really and he was saying of the entire interview how much he hated deep learning and how useless it was and then we we used deep learning to rescue his interview and he gave us deep deep deep deep deep that yeah she tell him yeah yeah and he gave us his permission to publish it that is wonderful so it’s quite ironic but no he always says it it’s wonderful for engineering but not a contribution to science yes sure yeah yeah he said that I like bulldozers too they’re good for clearing the snow but they’re not a contribution to science so so who else if I mean you’ve had a lot of people and I just enter Andrews one by the way it’s Andrew Lampinon yes he’s great so he’s great yeah it’s Andrew somebody I do work with quite closely oh wonderful so it was interesting listening to him because Andrew had quite a big influence on this paper by the way oh okay and I mean you know and but I think I might have had a bit of influence on him as well to at least think because that because that interview was just after he’d read and he read my paper he gave me lots of comments and we had a lot of discussion about it and and that interview looking at the recording date was was sort of just after this and it’s interesting I mean he was very circumspect and in some of the things he said yeah it was very interesting because he I think the influences maybe gone both ways yes which is which is nice I don’t know I mean I’m not I can’t be sure of that but I think there’s a huge similarity yeah I was thinking that actually just when you were speaking but it’s funny because you know because we’ve spent a lot of time arguing with each other about and you know I know I often feel like we’re we’re coming from very different perspectives on on this but in fact you know I think there’s there’s convergence really what are your areas of disagreement um well you see I I would have thought that Andrew would have been more on the side of you know we can do things without embodiment and without grounding or or or to kind of take grounding in a you know in a more liberal sense um because because some people take you know talk about grounding so they say well you know that large language models they are grounded prompt engineering is the process of using prompt prefixes to allow LLMs to understand better um you know so the the context and the purpose of a conversation in order to generate more appropriate responses what do you think is going on with prompt engineering? yeah well yeah so you so you let’s probably let slip a phrase there so the process of allowing the models to understand better is what you bet of course I don’t think guilty as charged I don’t think I don’t think that’s the right way of of characterizing it at all yes um so um I mean I think it’s the whole thing of prompt engineering is utterly fascinating and it’s it’s something that’s entered our world as AI researchers very prominently just in the last two years and it’s amazing I of course we have prompt engineering in the context of large language models we also have have prompt engineering in the context of you know the generative um you know image models as well like Dali and so on and um and and that’s really fascinating as well how by by you know engineering the prompt to be just the right sort of thing you can coach the the model into doing something which you know you might not otherwise do and it’s and it’s a great example of how alien these things are because if you were giving a human being the same instructions then you wouldn’t necessarily do quite what you do with with with either an LLM or an image model in order to get it to do the thing that you want it to do you have to kind of you have to kind of get into the zone with these models and and and and and figure out kind of what strange incantations are going to make it do the things that you want it to do now I think an interesting thing is that we may be looking at a moment of a very short moments in the history of the field where prompt engineering is relevant because if if if if language models become good enough then then you know we we’re not going to need to talk to them in this weird way you know engineer the prompt to get them to do what we want them to do it’s going to be you know it’s going to be a lot easier but anyway so maybe that will be the case I mean that makes a lot of sense that will be the cases as they get better but at the moment you know you you you you you can use a strange incantation like thinking steps and suddenly there are two language model will will be much more effective on reasoning problems than it was if you didn’t use the incantation thinking steps so that’s really fascinating so what’s going on there well I mean I think what’s going on there is that and we have to again bear in mind that what the model is really trained to do is is next word next word prediction but we have to remember that it’s doing next word prediction in this unimaginably complex distribution so and it’s and it’s and it’s it’s not just we have to remember that it’s not just the distribution of what a single human would you know the distribution of the sequence of words that a single human will come out with but of all the sort of text of of of you know millions of humans you know on the internet plus actually a load of other stuff like code and things which you know we don’t come out with in ordinary everyday language well people do it deep mind a bit but you know that’s deep mind so so so it’s it’s this unimaginably complex distribution and so I think what’s happening with with with prompt engineering is that you’re you’re sort of you know you’re kind of channeling it into some portion of the distribution so you’re you’re you’re queuing it up you know with with with with with the prompt and you’re you’re you’re this kind of context is putting it into some portion of this distribution and that is what’s going to enable it to do something different than it than it than it would have done if you had a difference out of words and and that would have put it in a different part of the distribution so you’ll find a finding the bit of this unimaginably complex distribution you’re finding the bit of it that you want to then concentrate on yeah so intuitively I agree because I think there’s two ways of looking at this so in I agree with you that they ask statistical language models I’m also a fan of the spline theory of neural networks which is this idea that you just kind of um testulate the ambient space into these little affine polyhedra and it’s a little bit like a locality sensitive hashing table but that’s quite it’s a quite a simple way of looking at it because you were talking about emergence before and emergence is all about this paradigmatic surprise a bit like the mind body dualism if you like there’s something that happens up here which is paradigmatically completely different to what happens down there so on the one hand we’re kind of saying oh they’re just simple interpolators or statistical models but on the other hand they really are doing something remarkable up here so so which is it which is it well I mean it’s both right so so so you know we’ll if we want to understand these models in a in a in a more scientific way which we surely do you know even if we’re not even if we’re not engineering them in an old-fashioned sense of engineering them but but rather they kind of you know emerge from the from the learning process but we still want to reverse engineer them to try and get as great as as as as comprehensive scientific understanding of these things as possible so so we want to understand it all these levels right we of course the foundation of that understanding is that we need to understand the actual mechanisms that we’ve programmed in there right so they were you know so you that’s essential you want to you know if you want to really understand these things you’ve got to understand transformer architectures the different kinds of transformer architectures you that you’ve got the you know what happens when you’d used kind of different parameter settings where this sparse or dense whether it’s in decoder only architecture or how you’re doing the tokenization how you’re doing the embedding when all of these things are essential to understanding you know and that’s all at the absolutely at the engineering level so you want to understand all of that but then we can do a whole load of reverse engineering at you know it’s another level and do the sort of thing that the people and thropic AI have done for example with with these induction heads and and and and understanding in terms of a transformer in terms of residual streams and induction heads which I think is fabulous work so that kind of thing is is looking it’s still quite a low level but it’s kind of the next level up and explaining a little bit about how these things work and work along those lines I think is like really essential and then the more complex these things are the you know the the hard the more we need to kind of ascend these levels of understanding and and and and you know and I hope that we can but I mean there’s no one that is the right one it’s you want to understand that things are all levels yeah different levels of description and you said something before which really interested me you said when the language models get good enough maybe we won’t need the prompts anymore and and I’d love to explore that duality because it’s a similar duality to how we talk about embodiment you know you can think of the language model being embodied in the prompt in some sense so maybe we’ll never get rid of the prompt but just to think about these prompts I think about them as a new type of program interpreter and there are some remarkable examples of scratch pad and chain of thought and even algorithmic prompting for getting insane extrapleted performance on lots of you know standard reasoning tasks yeah yeah and you know these these models are not touring machines they’re finite state with tomatis so they’re they’re limits to what we can do but I guess what I’m saying is the the prompt seems like it’s not going away anytime soon yeah so I think that I don’t think the prompt is going to go away but I think that the and I know who knows right but but but I think that prompt engineering as a whole kind of thing in itself you know may it may not be you know people talk about that as being a kind of a whole new job description here as prompt engineer and that so that that as a as a whole new job description I’m not quite sure how long exactly that will last because because prompting maybe just you know interacting with a thing in a much more natural language way in the way with with another person right so you know I don’t I don’t when I when I I don’t have to kind of think of some peculiar incantation in order to you know in order to get you know my colleagues to kind of help me on on something or to you know to cook a meal together with somebody we do I just we just use our natural kind of forms of communication and of course of course it does involve you know discussion and negotiation but it’s in this it’s just the same as we use of other humans right so so it may be that that rather than it being a peculiar thing in itself with all these funny phrases that just work for peculiar eccentric reasons that it may be much more natural amazing Professor Shanahan thank you so much for joining us indeed and thank you for the invitation it’s been lots of fun absolutely honor absolutely honor.

AI video(s) you might be interested in …