Examples include a training learning rate, batch size or architectural setting chosen for an experiment. These choices affect how learning proceeds and can influence performance, cost and stability.
Searching configurations requires a suitable evaluation procedure. A setting that works well on one task or dataset may not transfer, and repeatedly selecting against the same test can overstate the improvement.
ELI5
Think of hyperparameters as settings you choose before or around a learning exercise, rather than the answers learned during it.
For example, a team may compare several learning rates to find one that trains a model reliably without wasting compute.
How is a hyperparameter different from a parameter?
Model parameters are learned during training, while hyperparameters configure the model or the process used to train it.
Is the best setting universal?
No. Useful settings depend on the model, data, task, resource budget and evaluation conditions.
