ChatGPT Takes my Programming Machine Learning Course

ChatshipT is going to school and I’m the professor. Let’s see what grade ChatshipT gets in my course that I offer at Washington University in St. Louis in deep learning. And you’ll want to stick around to the end because I’m going to let you know what I think of this technology as a professor and what would happen if my students used it. I teach a class in deep learning at Washington University. Those of you that subscribe to the channel know all about it because I put all of the videos forward online here. There’s 12 assignments that the students need to deal with and then there’s also a Kaggle project. So we’re going to take these one by one and put them through ChatshipT and we’re going to see what happens. The first assignment is really pretty easy for human anyway. I want the students to get to know them because this is a virtual class. They come in campus at Worshew four times during the semester but the rest is completely online watching the same videos that I have on YouTube for this channel. So what they do for this is introduce themselves. I’m just going to take the instructions raw, copy and paste it into ChatshipT and see what happens. My major is computer science and I have a bachelor’s degree in the same. I have some experience working with machine learning specifically with supervised learning algorithms and have wow mind blown okay five out of five points for the very very first assignment. After this first assignment there are 10 programming assignments that they need to complete after each week. I’m going to take these one by one, copy and paste the instructions raw into ChatshipT, see what it does. Assignment one is easy peasy. You just put your key in just to prove that you can submit something and it got this one perfect five out of five. Assignment two asks it to do some very basic pre-processing on data both continuous and categorical. As you can see it did great on this five out of five. Assignment three asks it to summarize time series data like stock market so you would have the high low open close like candlestick charts are created from. It utterly messes this one up. It does not do the group rise correctly and ends up with the wrong number of rows and the wrong types of aggregations. Two out of five. Four asks you to construct two neural networks to fill in missing values for two columns that have those missing values. It does really surprisingly well on this one. It constructs both neural networks no reuse of code whatsoever but it does fail on an error because it fails to use dummy variables for categorical values. This is because it’s not really looking at the CSV files. It does not have the ability to go and look at anything beyond the pros that it’s given. I’ll give it a three out of five. Program five asks you to combine two CSV files together so that you have additional values and it also asks you to use separate CSV files for training and the ones that you eventually do the prediction on. Lots of steps coming together here and it did just fine five out of five. In this assignment I’m giving you two pictures and I ask you to make them square. You might have to put additional space at the top or at the sides of the image and I ask you to calculate the average color and use that as your fill color. It did really very well on this one. There’s multiple steps coming together. It figured out how to center the image exactly like I asked it to and it also figured out how to collect the average color. It didn’t just average all the RGBs together like sometimes students do which would result in a grayscale. However then it just dropped the ball and only did one of the two pictures that I asked it for and told you that you could do the other one and did not put it into the format needed for the submission. Decent effort four out of five. Assignment seven asks the student to use ten different images that I have of a beach and count up the number of people, handbags, other things that are on them. You’re using yellow V5 and it didn’t do well on this one at all. It used the wrong version of yellow and it just spat out some code for how to install yellow incorrectly. Zero out of five. Module eight I asked to do some feature engineering. I give it products with the three dimensions but I also say what type of geometric shape it is like sphere, cone, box and I’m figuring that what the student is going to do is go in and calculate these volumes add volume and then use that to calculate the cost that I gave you. Well it didn’t do that. It just calculated them like they were all cubes and then it encoded the type of object it was as dummy variables and maybe you’re leaving the feature engineering to the neural network. So okay, I can kind of see that not what I asked for but I can kind of see where you’re going but then it produced code that didn’t even work. You can see the error that it’s getting there, the shapes don’t align. That’s because it decided to drop the quantity field while it was training but not when it was evaluating. So two out of five. I always punish for code that doesn’t even complete. Nine I asked it to go in and look at these pre-trained models that are in Keras and just produce a table showing me the general layout of it. It didn’t do well on this at all. The code that it provided resulted in an error. It was assuming that that vector that it was dealing with up there was consistently always the same length which it was not. Again, I punish for non-working code and in this case one out of five. Sangmin 10 is another time series. This one is just simple time series prediction with LSTM or really your predictive model of choice there. It completely nails this one and gets a five out of five. The final project that I asked them right is just a write-up of five key points on a paper of their choice and I give you several papers to choose from. I just copy and pasted this right in. It saw that I was asking for a number of papers. It picked one and it addressed my five points. It’s a little shorter than I would like but it totally followed the directions. We are taking the Kaggle assignment out. ChatchyPT can’t work in a team or interact with the Kaggle website. So what is the final score? We’ll use the engineering school’s grading scale. Looking at the programs, ChatchyPT did not do so well. It scored a… To do better, ChatchyPT needs to use tools. ChatchyPT can write code but it can’t run it. It can’t look at data. It lacks the iterative process that makes data scientists great. I think ChatchyPT is stronger in the humanities. It’s writing abilities pulled its final score up to a C. What would happen if a real student basically used the ChatchyPT almost like a GPS where they’re driving but it’s really the machine calling the shots. What would I do? Well, first of all, they’d get exactly the same grade as ChatchyPT. So for graduate students, that’s probably not acceptable. Now, I’ve seen a slow progression of technology entering academics. I remember back in the day when I was young and dinosaurs were on the earth, teachers would get a little bit upset about if you used spell check in a word processor. I don’t know. I guess opening an addiction here made you a better person. But yeah, nobody cares anymore. Using Grammarly or something like that to correct grammar as you’re going through. I remember teachers getting hot and bothered about that one. Nobody really minds if you use Grammarly or some sort of grammar checking sort of thing. Now flipping over sort of into the corporate world, when you write, you typically use editors. If you write a book, you’re going to have an editor that it’s different when you’re in the corporate world. You use, you copy paste code from stack overflow. You do all these kind of things. So I’m not having an issue if my students are using Chatchy PT to complete the code. I’ve learned sort of what it’s good at, what it’s not good at. And I’ll construct the assignments to tighten the news around Chatchy PT’s neck a bit more. So you can certainly see some of its limitations and it goes off the rails really quite easily.