Is Apple’s M2 Max Good for Machine Learning?

So Apple has just released their new M2 chips and they appear to be quite powerful. Now the question is, would I buy one and what do I think I should recommend to you? First of all, let’s talk about who this is an obvious purchase for. So if you are convinced you want this, buy all means buy it. I’m not here to talk you out of how to spend your money. That is up to you. Second, if you are an Apple devotee who runs large models, large vision or language models, and need mobility as in you have to pick up your work machine and go to like a cafe or between home and a workplace, then by all means this is actually a really good, really good investment for you. It’s not a bad product by any means. However, if you are someone who is not devoted to Apple or you do not need mobility or you do need mobility and are not devoted to Apple, I don’t think it’s necessarily a great buy. I’m going to show you why in this video. So the first thing I want to take a look at are some benchmarks for the M1. Now we don’t have benchmarks for the M2 max as of yet, but we do have some independent benchmarks for the M1 max. This is from a user on GitHub and the reason I’m using this, you can take, you can have some issues with this that’s valid. The reason I’m picking this is because it’s very difficult to get unbiased benchmarks in the popular media. There were really only two sources that I trusted for GPU benchmarks and they were hard OCP and the tech report, both of which are now defunct hard OCP no longer produces reviews and the tech report is now a AI generated site that was purchased by some internet entrepreneur to make money with clicks, just total garbage, total garbage is quite a shame, they should have just shut it down. But I digress, it’s very difficult to get truly unbiased reviews from people that are receiving hardware due to their relationships with the underlying businesses. So that’s not where I choose to draw benchmarks from. This is a benchmark by a normal person, someone like you or I, that has access to both a 3090 and the M1 max. Now the reason the M1 max is relevant is because we can extrapolate based upon generational gains the performance of the M2 max. So taking a look at some performance on some basic models and I believe PyTorch, on say ResNet 50, they’re getting a performance throughput of 140 images per second consuming 21 gigabytes of RAM. Now this is why of memory, excuse me, this is why I said those that need access, those that are working with large models should consider this because the unified memory is a benefit if you are working with large models that require large amounts of memory. As of yet, there are no comparable GPUs that are going to give you access to more than 16 gigabytes of V RAM in a mobile form factor. So if you need that, then this is the only game in town. And the performance is pretty good. Performance per watt is what I would expect it to be. So I don’t think it’s a terrible purchase in that case. But when you compare to the 3090 down here, you can see that it’s 1100 images per second. Now that’s not surprising. It’s something like what is that seven or eight times, eight times of performance and also about eight times the power of raw slightly more. So it’s a little bit less efficient for that speed gain. So we know that the M1 max is about an eighth, you know, the performance of a 3090. And we know that the generational uplift is going to be something like between 30 to 40%. So if you take a look at this article from Macroll, it says, an Apple’s benchmarks again, I take those are the grain of salt, but for the purposes of argumentation, let’s just take them as is and give them a bonus, say they’re sandbagging. So it says M2 max tells a 31% improvement over the M1 max and the Vinci result. Again, that’s not that’s not machine learning. So I don’t know and says it’s faster than i9 MacBook Pro and some other tasks and with rendering in cinema 40, it’s 30% faster with the M1 max. So 30%. Let’s assume that the sake of argument round numbers, it’s 50% faster in machine learning tasks. I don’t know where that’s going to come from, but let’s just assume it’s, you know, 30, excuse me, 50% faster. Taking a look here, this eight to one then goes down to about five to one ish approximately comparing the 39 to the M2 max and the same benchmarks. So about five X to performance and I’m assuming performance per watt is going to be better. So it’s going to have the same peak power. So it’s going to get better with respect to performance per watt. Okay, all well and good. So if you have a use case for this, if you need that 21GB of memory, this is the only game in town, cool, go buy it. I don’t have a problem with that. I’m not a zealot. I’m not, you know, I don’t like Mac products for myself, but I don’t care if you buy them. However, where things get a little dicey is if you take a look at other types of models and I know that for what I do here on this channel, deep reinforcement learning, you don’t need huge amounts of V RAM. You need like a gigabyte or so for the models that we use, maybe two at most, if you’re running something really, really complex. So about a gigabyte of V RAM, so pretty much any modern GPU can handle that. But then you see stuff like 16 and, excuse me, 13 and 14 gigabyte. So something to do to comfortably fit in the envelope of a 16 gigabyte GPU. And the performance, you know, varies from, you know, seven to eight X or whatever. So it’s significantly slower than the 3090 as you would expect, even accounting for the generational uplift with the M2 max. Okay. So then that raises the question. Suppose you don’t need these huge models. You have slightly less stringent requirement on your V RAM on total memory. What should you get then? Now if mobility is a concern, if you really need mobility, but you’re not devoted to Apple and you don’t need the really large models, then I would actually consider something like a last gen laptop. So looking here, oh, and the other thing to consider is price. So if you take a look at a reasonably specced 16 inch MacBook Pro with the M2 max, with 32 gigs of unified memory, so enough to comfortably fit the ResNet model and a terabyte of SSD storage, you’re looking at a price of $3,500. Now, that is a significant chunk of change. If it makes you money, it’s a solid investment. Again, if you need it, you need it, buy it and move on with your life. Don’t even lose sleep over it. But what you can do then is take a look at some comparable products that are not in the Mac ecosystem that will offer a similar performance. So are better performance because they’re based on Nvidia GPUs. So a last gen laptop here, so this MSI from New Egg runs $3,400. It’s got a one terabyte PCIe, so it’s a slightly faster SSD drive. Does that really matter? Not so much. It’s not so critical, but it is faster and faster is always better. You do get the same amount of RAM, and this processor is a couple generations old, but offers 16 threads as opposed to 12 cores. Now, as far as I can tell, the Apple chips don’t have simultaneous multi-threading, so when it says 12 core, here it really means 12 core. It’s not 12 core times 2 for 24 threads. It’s just 12 core. So you get 16 threads versus 12. Okay, cool. But then scrolling down to the GPU, this might be a little bit hard to read. The word of the GPU go. It is down here. So it’s a radio, excuse me, G43080 with 16 gigabytes of VRAM. So it will comfortably fit everything except the ResNet models. And the 3080 laptop GPU is going to be around, by my guess, it must twice as fast as the M2 max, maybe two to three times. This is going to be around 60% or so of the 3090. So it should be sufficiently fast to be faster than the M2 max for the same sort of price envelope. Now, of course, power efficiency will not be as great on this because it’s a dedicated GPU. So if power efficiency really matters to you, which I don’t honestly think it does, but if you want to lie and say that it matters to you, then by all means buy the Apple product. So for a few hundred dollars less, you get something that’s going to be nominally faster. I’ll be at base on Windows. I’m not really a fan of Windows. I would personally wipe the hard drive and put Linux on it, but that’s just me. So then that raises the question, OK, so if you are in the Apple fan base, you are dedicated to the Apple ecosystem. You love it and you need it. File means buy it. If you are not dedicated to the Apple ecosystem, but you do need mobility, then perhaps a Windows laptop is something to consider or better yet. Perhaps even a Linux laptop from a company like System76 or even Dell would be something that would appeal to you. Now if you don’t need mobility, which I don’t, you can see here, I use a desktop. I’m in camp desktop. And firmly, I don’t really like mobile form factors. I don’t think they do anything any better other than be moved around. And I don’t want to move a computing device around, you know, if I really want to do something I have a phone, otherwise I don’t need mobility in my computing device. However, what is the better option? So I have specced out a high end desktop on new egg. This is using current prices as of February 1st, 2023. These may be lower or higher in the future. Of course, we have just come out the other side of the mining boom and GPU prices have finally started normalized and low and behold, I looked this morning, could actually see a RTX 4090 for sale on new egg and in stock. Now probably by time this video goes live, it’ll be gone, but I did find other options for 50 to $100 more on other websites. So if you look around, you can purchase a 4090 and if you live near a micro center, then you can just go pick one up most of the time. But taking a look at what I have specced out here, you have a motherboard GPU, CPU RAM, monitor, fan, case, power supply and one terabyte NVMe SSD. And it all comes out to $3,600 excluding tax, of course. So for about $100 more, you get a significantly better desktop system than you do with the Apple. Now, you can install any operating system on this. You could, if you were intrepid enough, try to do a hack and toss with it. I don’t know. I don’t actually think that would work for you, but I’ve never tried it. So I don’t know, but it may be something to consider. I would run Linux or BSD or if you really must, then you can run Windows on it. Of course, that’s always an option, just factoring a little bit of money for the operating system. But the point here being you get an AMD high-end CPU 16 core 32 threads. 32 is oftentimes greater than 12. You get 32 gigs of RAM, a 24 gigabyte GPU, which will run those resonant models. You get a high-end motherboard, you get a decent monitor. I didn’t price out the monitor too much. That’s up to you what you want to spend on a monitor, but you get a high-end air cooler and a very high-end case and power supply and a significantly faster SSD to boot. So for $3,600, you get the top of the line desktop that’s going to last for several years. And that’s the camp I’m in personally. One of the cobit is I don’t necessarily endorse any one of these specific parts. I’ve just thrown these in here for pricing comparison. So if you buy it and you say, hey, this board sucks, you have the motherboard sucks. I didn’t say go buy this motherboard. I just said for $300, you get an AM5 motherboard, where you’re 79.50X CPU. So other thing to consider though is that Nvidia is dropping their new mobility chips for the 4090s for the 4000 series. So the 4090 is on sale. On Amazon, it’s currently sold out. But the price is 38.99. And this will come with comparable specs, 32GB of DDR5 RAM, a 2TB NVMe drive, so pretty fast. 17.3 inch chassis. So same size as the MacBook basically. Or a little bit more money and the 4090 mobility is going to be significantly faster than the M2 Max and pretty much any task. It will consume more power. It’s a few hundred dollars more expensive, but if you can afford $3,500, you can probably afford $3,900. Now that’s if you’re agnostic to which platform you want to run on, you know, Mac versus Windows. And you need mobility. This is a great option. Or you can save a few hundred dollars, get something that’s last gen in terms of GPU and a couple of gen behind in CPU, which isn’t as important given the slow progression and hardware requirements for on the CPU front. But you still get more threads than you do in the Apple case. So I hope this was helpful. That’s how I take a look. Oh, there is one other thing. And this all came about because someone in my Slack group, so for my students on the neural net academy were in a Slack group, they said, Hey, what about the new Mac mini with the M2 Pro? And my gut reaction was no, it’s not really that great. And I still, I haven’t really warmed up to it after looking at any of this. And the reason is that again, if you’re not within the Apple ecosystem, you know, you don’t have to have that for the price. It’s really not that great of bargain, right? Because you can buy, you know, a 4070 Ti or you could buy a 3070 Ti an old 3080, something like that for seven, 800 bucks and build a system around that of comparable quality. And this is kind of the worst of all worlds, right? Because it’s, it doesn’t have any, any peripherals. You don’t have a monitor. You don’t have a mouse or keyboard. So it’s not mobile unless you’re going to lug around a monitor and mouse and keyboard, in which case. I don’t know what you’re doing. That’s not really mobile. So it’s not really mobile. And if you don’t need something that isn’t mobile, then really the only reason to get it is if you’re in firmly in the Mac camp, in which case, hey, it’s not a bad product. You know, I’m not here to hate on Mac. I just don’t think they provide very good performance per dollar. I’m not firmly against Apple. It’s not something I choose. It’s not an ecosystem I choose to buy into. It’s, it’s antithetical to my personal values, you know, the whole walled garden stuff isn’t for me. But if it’s for you and you have done your research and believe this to be right for you, then by all means buy it. I’m not going to sit here and bash Apple fans because, well, that’s a stupid thing to do. So I don’t think the performance per dollar is very good. I think if you don’t need mobility, if you don’t need large models and you are not in the Apple ecosystem, then a desktop is going to be a far better option. If you do need mobility, but don’t need very large models and are not married to the Apple ecosystem, then a Windows based laptop with a dedicated Nvidia GPU is going to be a significantly better performing option in terms of absolute performance, though not performance per watt of course. That is all I have to say on that in the coming days, I’m going to make another video on my thoughts around the current state of what type of system to buy. This isn’t like a system buying guy. This is just my thoughts on the M2 chip. Any, you know, disagreements, leave them down below. I suspect there will probably be some love done my best to not step on anyone’s toes here. So disagreements, comments, questions, suggestions, leave them in the comments down below. And to see more stuff like this, hit that subscribe button. And I’ll see you in the next video.