AI learns to play 2048

What’s up guys, today’s video is on 2048. Where’s the second to nick my machine video? You promised us! Well honestly, I couldn’t be bothered doing it, so if you guys want to see that then comment below, but I want to do this more, so I did it. Anyway, now that’s out of the way, let’s talk 2048. I’m sure you guys know what this is, it’s just where you move tiles around and get a high score and whatever, so we need a goal. And what’s better goal than 2048? Let’s go for the stars, let’s get that 2048. Alright, let’s begin, same as always, first of all we need to create the game. So to start off with, let’s get some squares on the screen. When a new tile is created, there is a 90% chance that it is a 2, and a 10% chance that it’s a 4. Alright, boom! Yeah, got some squares on the screen. Try not to subscribe too hard, the button can only take so much abuse. Anyway, we need to get some actual gameplay happening, so I created a player which contains all the tiles, I call them tiles now. When the game starts, it has two tiles in random positions. Now let’s add the movement. In 2048 you can move in four directions, every tile is moved in the direction until it hits the wall or another tile. If two blocks collide which have the same value, they do a quick fusion dance and create a new tile with twice the value. Okay, here we go, this should be working now, let’s see what happens when we press the up arrow and they’re gone. Let’s try that again, yep, they’re gone, fantastic. Alright, so we need to fix that. Right here, there we go, now let’s stick around. This is still wrong, but we’re getting it. Okay, so the problem is we need to get the correct order in which we move the tiles up. We need to move the ones closest to the top first, let me explain. So we have two tiles like this, if we move the bottom one first then we’ll move up one space and stop because it hits another tile, I haven’t implemented any of the fusing action yet, so it just stops. Then the tile at the top moves up until it hits the top. So we need to move the top tile first so it works properly. Okay, ready, wait for it. Oh yeah, that’s how it’s done, baby, okay settle down. Now we need to add a new tile every time the player moves. This was actually way more difficult than it should have been, but I got it. So yeah, every time you move a new tile’s added, fantastic. But this is still far from the 2048 game we know and love because we need to implement the fusion thing I’ve been talking about. It really wasn’t all that difficult, so here it is. Now the game is done, but it still looks like trash. All the movements are hard to follow and the blocks are the same color, so we need to make it look a bit prettier before we can try to crack it. So I made these changes and I’m actually pretty happy without turned out. Here it is, pretty sexy, right? So now that this game’s done, let’s do the AI. So the first generation is set up and a problem immediately became apparent. In 2048 you can get pretty far by just pressing random keys. This means that the evolution is much harder because those which are terrible don’t just die off since the difference between random movements and intelligent movements is very small. So a strategy of repetitively pressing the left button emerged. If they couldn’t go left and they went up and if they couldn’t go up then they went down. That’s not hard. The strategy was actually rather successful, reaching the 512 tile by the first generation. By generation 3 a similar tactic emerged, but went right instead of left. This was somehow much more successful, reaching the 1024 tile. After this I was confident that we could reach the 2048 tile in no time. But well I mean, you’ll see. The next improvement wasn’t until generation 29, which is a bit of a step, and it only improved a little bit, getting just over the 1024 tile again. So I let it run for a bit, and then a bit more, and then some more. Anyway the next improvement wasn’t until generation 247. So there was obviously an issue. For newer evolution to work it requires gradual increases over time. Think of it like the baby step approach. This is not that. This is more like bloody steroid baby steps. But anyway I tried writing in a few more times with different inputs and for much longer but they all had the same result. So it’s got me thinking that neat wasn’t actually the best choice for this problem. But you know me, I’ll never take no for an answer. I’ll keep work until I die. When the going gets tough, the tough give up and try something else. Yeah I gave up. Sue me we’re not all perfect. But do not fret, the goal was 2048 and I’ll be damned if that doesn’t happen. So I tried a much more classical approach to AI. Now this newer evolution fancy stuff, just some classic searching algorithms. And well I know the word algorithm scares you guys because then would you hear it you think it’s not AI. But you know that’s okay. Everyone has the right to an opinion but not every opinion is right. Okay now that that’s out of the way. So first I need to introduce you to the concept of game states. Game states are moments in time for the game like this or this. Well my algorithm will do is start from the current game state which is just what the game currently looks like and construct all possible game states which you can get to from this one. Taking into account all possible moves and the randomness of tiles appearing. Thomas here, my man Thomas looks five moves into the future. At all the possible game states then finds the best one and then takes the moves which are required to get there. The interesting bit comes from how to tell which game states are the best. To do this we create a function which calculates a single number representing the value of that game state. Kind of like the score. For example game state which you die would have a value of zero and one which you got the 1024 tile might be like a thousand. You might be tempted to just use the score as the game value. However there is a problem with this. The value of each game state doesn’t take into account the potential of that game state. Consider these two game states. While they would have the same score the one on the right is far more valuable as it has the two highest tiles next to each other. So the value function needs to take into account adjacent blocks of the same number. One final thing that I added before we run it is strategic to keep the highest block in the corner with the next highest next to it. That way when new tiles spawn they spawn far away from the larger ones. This basically separates the lower tiles from the higher tiles. So we’re going to give a value boost to game states which have the highest tile in the top left corner. Very nice. Okay let’s see if he can do it. Yeah Tommy’s a problem child. I’ve no idea who’s doing it. He’s going to die. He died. Fantastic. That’s why you keep it top left Tommy. Yes 512. Quarter the way there. Your bat is good as repetitively pressing the left gate. So it’s actually quite stressful. Oh yes. Yes. Oh my god. Way close. Whoosh. Presses on Tommy. Keep it top left. You doing well. Oh my god. No. No Tommy. No. You’re embarrassing me. Alright Thomas. This one will be the one. You can do it. Yeah so Thomas kind of sucks but honestly I’ve had a bad enough of this project. So we’re going to sit here and get a white until he bloody gets it. Oh Thomas. You’re so close. Thomas you dumb mother. So while we’re waiting for Thomas to get his act together. I’m now on Twitter. Yeah. I don’t really know what to do that but I’m there. Tommy no. We bloody time. Goon. Aaron. Come on, come on, Tommy. Tommy, your baby! He’s done it! I can go to bed! Yeah, he just keeps, yeah, okay, no, he doesn’t. He dies. Did well, Tommy. Did well. So the theme for today was failure. There were issues with my first approach to creating a solution, and Thomas was certainly no stranger to failure. Damn it, Thomas. But this isn’t necessarily a bad thing as Brood.org puts it in the eight principles of learning. The best learners allow themselves to make many mistakes along their journey. Hell, 90% of what I do here is fail. Let’s tell most of my AI’s work, they fail, and they learn making them slightly less likely to fail. Brood.org is not a Bach. They actually practice what they preach, giving you valuable feedback every time you don’t get a question right. So you actually learn from your mistakes, not like here at university, whether just like, wrong, you get a zero, and then you’re like, but why? Shut up. What do you think you’re here to learn or something? Go home, you bloody failure. I’m sorry, I’ve just finished my uni exam, so still a bit salty. Anyway, Brood.org understands that failure is an important step in learning. So head over to Brood.org slash code bullet, check out more of their learning principles, and get your hands on a bunch of their free courses. The first 200 people to use this link will get 20% off an annual subscription. So guys, get failing, and maybe one day you can reach the elusive 2048. Drink with food. I’m thinking I’ll learn tickling. And potentially every second. compromise is booster, is one thing that needs to be easy.