Speculative decoding uses a smaller model, auxiliary layer, or other fast mechanism to draft likely output tokens. The main model checks those proposals in parallel and accepts the ones that match what it would have generated itself.
The method can reduce latency without changing the target model's output distribution when implemented correctly. Its benefit depends on proposal accuracy, verification cost, hardware utilization, and serving integration.
