Theo Browne explains how shared memory in Apple computers can support local inference with models that exceed the memory available on many graphics cards. He demonstrates loading a larger model and discusses the tradeoffs compared with a discrete GPU.
He compares Macs with alternatives including Framework Desktop and Nvidia DGX Spark, and discusses storage performance, rising memory costs and their effect on computer prices. His purchasing suggestions reflect the prices and availability at the time of recording.
Watch on YouTube
