A load balancer decides which eligible worker receives a request. It can consider capacity, current queues, location, capabilities and reusable state instead of sending every request to one worker.
Distribution needs to account for failures and uneven workloads. A routing policy can improve resource use without guaranteeing that all demand can be served, and extra retries can worsen overload if they are not bounded.
ELI5
Load balancing chooses where each incoming job should go. The goal is to make sensible use of available workers rather than leave one overwhelmed while another has room.
For example, an AI service can send a request to a less busy server even when it is farther away. The extra travel time may be smaller than the wait at the nearest overloaded server.
Does load balancing always choose the nearest server?
No. Capacity, waiting time and task requirements can make another eligible server a better choice.
Can load balancing create unlimited capacity?
No. It distributes available resources, so demand can still exceed what the system can serve.
