Hannes Rudolph and Patrick Erichsen talk with a guest from NVIDIA about running AI models locally, including privacy-sensitive workloadsData privacy governs how personal, confidential, or sensitive information is collected, used, shared, retained, and protected in AI systems., routing between local and cloud modelsAI model routing sends each request to a model chosen for that request's complexity, cost, speed, privacy, or other requirements. and unified memoryAI compute infrastructure is the hardware, facilities, networks, power, cooling, storage, and software used to train and run AI models.. The guest describes NVIDIA hardware and models while acknowledging a commercial affiliation.
The conversation examines hardware costs, the limitations of local modelsA local model runs on computing infrastructure controlled directly by the user rather than only through a remote model service. for concurrent tasks and possible uses ranging from code exploration to home automation. The speakers recommend assessing individual needsModel selection chooses the AI model whose capability, quality, cost, speed, safety, and operating constraints best fit a task. and researching devices independently.
Watch on YouTube
