Computerphile: Deadly Truth of General AI?

In a very basic sense, if you’ve got a general intelligence, which therefore has preferences over world states and takes actions in the world to change the world, we have to make sure that its preferences are aligned with ours, that it wants what we want, because otherwise it’s going to try and do things that we don’t want it to do. That’s the basic idea. Sci-Fi comes to the fore, but we’re talking things like Terminator Matrix, that machines take over the world, I for one, salute our new robot ever lost. It makes it difficult to think about the problem, right? When things happen in fiction it’s generally what would make a better story, rather than what would actually happen. And I think a realistic AI takes over the world type story, it might not be any fun to read. So on the one side, this stuff is difficult to think about because of fiction, because we’ve all been exposed to something similar to these ideas before. The other side that makes this difficult to think about is anthropomorphism, because we are talking about general intelligence, so we’re going to compare it to the examples of general intelligence that we have, which is human minds. And human minds and artificial general intelligences need not be anything alike. In the same way that a plane is not similar to a bird, a supersonic fighter jet is a threat to you in a way that no bird is. It’s not a useful comparison to make, in fact. But when you say, oh, it’s a thing, it has wings and it flies and people who don’t know anything about planes and need to go to birds. Presumably, a machine could be much more selfish than we can ever imagine. Absolutely. The space of minds in general is vast. I like this because we’ve already talked about spaces so I can do this. If you take a space of all possible minds, it’s huge. And then if somewhere within that you have the space of all minds that biological evolution can produce. And that’s also huge. Somewhere within that you have the space of actual minds that exist, which is much smaller, but still huge. Within that you’ve got human minds. And there are tiny, there are minuscule dot on a minuscule dot on a minuscule dot of the actual possibilities for intelligence that exist. And a general intelligence that we create is from a completely different part of the space. And it’s extremely tempting to anthropomorphize more so even than in other contexts because it’s a thing that’s demonstrably intelligent that makes plans that takes actions in the real world. But it need not think anything like us. And it’s a mistake to think of it as basically a person because it isn’t one. So there’s actually a really good example that we can use. It’s sort of a thought experiment. This is not a machine that could practically speaking be built. This is an example of artificial general intelligence, which is specified in overview. And it gives you something to think about when you’re thinking about artificial general intelligence is that makes it distinct from a sort of anthropomorphized human type intelligence. So the story is there’s a stamp collector who is also an AI programmer. And he decides he would like to collect a lot more stamps. So he’s going to write an AI to do this for him. So he builds a machine. He has some startling insight into general intelligence. And he builds this machine which is connected to the internet. So the rules for this system are pretty straightforward. The first thing is connected to the internet and it will send and receive data for one year. So he’s given himself a one-year time window within which to collect stamps. The second thing is it has an internal model of reality or of the universe. This is the thing that’s a bit magic. We don’t really know how to build an accurate model of reality. The point is this allows it to make accurate predictions about what will happen if it does different things. The third thing is for every possible sequence of packets it could send, it uses its model to predict how many stamps it ends up with at the end of that. And then the fourth thing is it outputs as its actual data to the internet which ever output it has predicted will produce the most stamps. You can see here that this has all the properties of a general intelligence. It has an internal model of reality. It has a utility function or an evaluation function which is the number of stamps. And the optimization is extremely simple and like so much in computer science the the simple things to specify are the hard things to compute. It looks at every possible output of data. In other words every every point in that space and it picks out the highest one. So this is the kind of magic intelligence that takes the entire space at once finds the highest point and says that one. Which means it’s an extremely powerful intelligence. It is you could say it’s extremely intelligent. The question is how does this machine behave? Well we can look at certain possible sequences of outputs and see how they fare in its evaluation criteria. First off the vast majority of output sequences are complete junk. It’s spewing random data onto the network, nothing happens of any consequence, no stamps get collected. Those are all rated zero. But suppose one of the possible sequences sends a request to a server let’s say eBay that results in a bid on some stamps. Right? When that happens there’s a thing of 20 stamps so that output is rated 20. This is the kind of thing that the stamp collector machines creator was expecting to happen. So then that’s good. 20 stamps. Suppose it could do that lots of times. It could send out bids for example to 30 different stamp collectors on eBay and by 30 sets of different stamps. And that’s even better, right? That’ll be rated even higher. But the thing is that particularly highly rated options in this search space are probably things that the stamp collecting devices creator did not think of and did not anticipate. So for example when he made it he will have presumably given it his credit card details or something so that it could engage in these bids. But ultimately it’s searching every possible sequence of outputs. It needn’t use his credit card. It needn’t use money at all. There’s a huge variety of things that it could do here. So it might send out an enormous number of emails to all the stamp collectors of the world and convince them through persuasive argument that he is opening a museum and wants to exhibit their stamps. It may build a fake website for that. Whatever is necessary it can predict the world. It has an internal model of reality. That internal model of reality includes the people. In the same way that it’s modeling people to understand that if it bids on this offer then a human being will mail the stamps to them. They understand that this email might get more people to send stamps. For example that’s something. But then what exactly is a stamp? How is this defined? What counts as a stamp? If it’s already written a sequence of outputs that collects all of the stamps in the world you’d think you’re done right? You’ve built this machine suddenly it’s collected all the stamps in the world. No there could be more stamps within a year. We’ve got time to print more stamps. So maybe it hijacks the world’s stamp printing factories and puts them into overdrive producing stamps as many as it possibly can in that time. Or perhaps it writes a virus and it hijacks all the computers in the world to get all of the printers in the world to do nothing but print stamps. That’s even better right? The highest rated outcomes for this machine are not good for people. There comes a point when the stamp collecting device is thinking okay what are stamps made of? They’re made of paper. Paper is made of carbon and hydrogen and oxygen. I’m going to need all of that that I can get to make stamps. And it’s going to notice that people are made of carbon hydrogen and oxygen right? There comes a point where the stamp collecting device becomes extremely dangerous and that point is as soon as you switch it on. So this is just a thought experiment example where we take a sort of maximally powerful intelligence and see how it behaves so that we can think about how powerful intelligence is behave and most importantly how they are not like people.