Less sensitive parts can use smaller number formats to reduce storage and computation. Parts where rounding would hurt the result can use more precise formats.
The choice depends on the workload and hardware. A comparison should check quality as well as the speed and memory improvements.
ELI5
For example, a model might store most weights with less detail while keeping a few sensitive sections at higher precision.
This can let it fit on a smaller device while preserving accuracy where rounding would otherwise cause problems.
What does precision refer to?
It describes how finely a number format represents values, which affects the rounding incurred when a value is stored or processed.
Will every device run mixed precision faster?
Performance depends on which numeric formats the hardware can process efficiently and how much conversion is needed.


