Computerphile: Deep Learning •

Tell me about deep learning. Maybe it gets slightly Matthew, but we can just go after with an example to see what the things are. But one typical example is if someone wants to say automatically what is the price of a property. So there are certain variables that are important for property, you know, X1, X2, to whatever XN. You know, this might be the square meters, this is like a year of construction, whatever else, right? And then the most simple way of finding an estimated value is to say, okay, I’m going to do this. A1, A2, HN, plus, you know, usually we have a bias term, something like that. This gives you the estimated price. You can represent it like this, right? You have the input variables and then you have your output variable and then you do this, meaning that you have this value, you have an A1 here and AN here, and all of this is summed here into this summation. Okay, so for instance, the larger the building, the higher the number, maybe the newer the house, the more valuable it is or whatever. So this is just a weighted combination of 13 factors, right? So it all comes out a big number, right? Hopefully you will have more or less accurate weight of describing the prices. So how do you learn this thing? So you can just, you know, decide yourself, put the seven here more or less, what comes out makes sense, right? But usually you find, say, 100 examples, right? If you’re interested in doing this, maybe you have a backlog of houses sold, right? And then you more or less plot the value in order to visualize it, we do it with only one variable, which is square meters, right? So you have one example here, another here, right? And you have this cloud of examples, right? So this means like each one of these x-axis, you sold a house that has, you know, this number of square meters for this price over here. And then you try to more or less feed that, right? So you can say, I’m going to do, like, straight line, right? That will tell me, you know, what is the relation between the square meters and the value, yeah? So now tomorrow, someone tells you, you know, I have like this 80 square meters house, what is the price, right? And then you come here and you say, okay, the price is this, yeah? Which is the best line, you don’t know, right? That is the question. You say maybe a line is not a good thing, right? So you will use a quadratic function, right? Because, you know, like maybe larger houses, they’re more, they’re not so easy to combine, and usually have higher end, so, you know, things will not grow linearly. So it’s a curved line for those who are not mathematical, it’s a curved line possible. It’s a curved line that will involve more variables, right? And this means that you can feed things that are not so straight, right? So you have more flexibility in what you can model, right? This comes at the cost of having more variables to describe the model, right? And there’s always this type of relation, right? You have more power to represent something, and then you can capture fake correlations, right? So how to, which kind of model is best, this is something that is not clear at all, right? There’s not like just throw as many variables as you can and will feed better, right? Because you will be doing something that is called over-feeding, which is, you will be over-estimating how well you do things, right? So maybe if you put enough variables, right, and you have a set of points, right? And then you will do this, right? And you say, okay, I exactly predict everything, right? But then, you know, the reality is that at some point you will get something there and you’re totally wrong, right? So many parameters make things hard to optimize, right? So in the flat structure, you have all of these, and then you have this variable that is disamation, right? And this is your output, right? And then you can do, as opposed to this, a deep structure, right? So it’s a flat structure, it’s just kind of traditional machine learning, is it? Yeah, yeah. Neural networks is something that has been around for a very long time, and neural networks can be either shallow, right? Yeah, shallow is better than it is, a shallow structure, or it can be a deepest structure, right? Shallow learning, we add everything together, we come up with a number, that number gives us how good is this, or how bad is that? Yeah, deep learning, other stuff’s going on, right? Yeah, so there’s some variables in the middle, right, in between your input, right, the values that you’re able to measure, like the size of the flat or the neural construction, and then something happens that tells you some intermediate variables, right? And then from this intermediate variables, you will get the output, right? You will get the prediction, or maybe you will put yet another intermediate set of variables, and so forth, right? So the idea is you have some compound concepts, right? That might be good for predicting what you want to know. You could have a direct prediction of, you have this kind of structure that will tell you what is the price, yeah? But maybe the most relevant variables in order to decide what is the pricing is not one of these things that were inputting. You might have to find some concepts that relate to a subset of this in a certain way, and that this is the real important thing, right? So for example, you might want to have either a very large house with several bathrooms, right? Or you might want to have a small house and having one bathroom is not a problem, right? So this type of concept is a combination of more than one, right? You cannot decide having one bathroom is bad, right? Because it depends on the number of square meters of the house. The idea is that with deep learning, this thing here will not be the price. This will be this intermediate concept, right? And you will say, you know, for example, I will create a concept that you have bathrooms and number of rooms on one variable. And then these other things in here will not really matter, yeah? And then you can have something here that is basically the square meters, you know, like it’s an important variable independently of anything else, right? So we will keep this, this is the only thing that will matter for this variable, right? You know, maybe the year is a really good thing to have a new flat, right? But maybe if the property is more than 100 years old suddenly, you know, it becomes really good, right? And maybe you want to have a concept that says, you know, is this a new property or you will have yet another concept that this is a historic property, right? And this will depends on the year, but the type of relation will be different. Because suddenly it’s not that all this bad, all becomes good, right? So you have to have this set of intermediate concepts, right? In a way that now you can use them to decide on the pricing of your house. And these probably will be more meaningful, more representative of what people seek than just having the pure basic measurable things in isolation. So once you mix and combine some of these will give you attributes that are better in order to predict. But you can just do it again and again, right? So there’s no reason why you have to combine this thing, right? Only once, right? And suddenly you want, I don’t know, for whatever reason you might have a lot of market for very old houses that actually have an even number of rooms and bathrooms, right? And then you create yet another layer, right? So instead of having these connections directly to the pricing of the house, right? You have yet another connection, right? We said that we had a house with the tar is really old, so it just went up in value because it’s a historical building. And then you have a very good configuration of rooms and bathrooms, right? So this means that for example, this variable in here that captures whether it’s a good balanced rooms, bathrooms will have a high value, right? We say that this variable is active. This will mean that you have this variable active, you have this one that was the one representing whether the building was becoming historical, right? And with these two things having a high value, then you will have a high value also for this variable over here that will mean that you get a really high selling price in the end, right? Instead, if you have a really bad combination of room, bathrooms, right? This variable in here will have a very low value, and then instead of this variable, you will have the other one being active, and you will have a low selling value or lower than the other case, right? So this represents the kind of choices that actually you can make when you reason about things, right? So what do you want? We want not only square meters, rooms, bathrooms, you actually take decisions based on the combinations, right? So you will say, well, you know, it is a large flood, but the distribution is really good, right? So this is the kind of combination of two variables, right? Having good distribution in large, maybe if the flood was large enough, you wouldn’t care so much about the distribution, right? And you can keep on having this hierarchical grouping of different variables, right? And the good thing about machine learning is that in the same way that you reason try to find a correlation that would explain that these things together are not good enough. So if you have things together, make sense, the machine is able to find grouping of different variables in a way that you basically have no idea what it means, right? But the machine knows that this is useful to take a decision, right? So the machine is trying out various different things, and you’ve told him what the endpoint is, and it worked out what gives you the most consistent and accurate. So the machine will be able to say, okay, in order to get an accurate prediction, you have to group these variables, right, and create this kind of decision, right? So if you meet all these conditions, then this thing will have a high value, right? And whenever this thing happens and some other decisions like that happen as well, and they have high values, right? Then this thing will happen, right? And you can keep on going really deep in the sense, right? But the idea is that, right? You will take decisions based on these decisions based on the input values, right? So you will combine your inputs into some decisions that are intermediate, they’re not really the pricing, right? And then you will have another layer that will decide on the price later on. If it tolerates you and gives you some idea of what’s happening, and you can sort of see where I must do what I’m talking about. And this can represent the other paddles. So we’ve got two objects with the identical interfaces. One to represent the paddle on the left, one to represent the paddle on the right.