Google AI Edge engineer Cormac Brick contrasts cloud inferenceInference is the use of a trained model to produce an output from supplied input. with local intelligence that can work offline, preserve data on the device and provide predictable latency. Memory cost and the wider range of target hardware constrain deployment: fitting model weights is only part of the budget once the runtime, cache and operating system are included.
Brick compares general small models with much smaller specialists for speech, vision, embeddings and function callingAI tool calling lets a model request a structured operation from software instead of only returning natural-language text.. He reports hardware-dependent performance and demonstrates why merely running a model is not enough if an interaction feels too slow. The transcript's contradictory tiny-model parameter phrase is not adopted as a technical specification.
Task-specific fine-tuningFine-tuning adapts a pretrained model by updating its parameters on examples chosen for a particular task or subject. with synthetic dataSynthetic data is information generated to represent examples or situations rather than collected directly from the events it describes. is presented as a route to compact, responsive features. Examples include voice commands mapped to a fixed set of functions and offline dictation using separate speech-recognitionAutomatic speech recognition converts spoken language into text or structured linguistic output using signal processing and machine learning. and text-polishing models. Brick treats easier customization and faster visual perception as future work; reported reliability remains specific to the demonstrated tasks.
Watch on YouTube




