Instead of requiring a block to learn an entire mapping directly, the architecture provides a shortcut alongside the learned transformation. This supports residual learning and can make deep networks easier to optimize.
For example, a block can preserve the incoming representation while adding a useful correction. Addition requires compatible shapes or a suitable projection; the shortcut alone does not guarantee that training will succeed.
ELI5
A block can keep the information it received and add a learned adjustment.
For example, think of revising a draft by adding a correction instead of rewriting everything. The original path remains available alongside the change.
Is the shortcut always an unchanged copy?
No. A projection can adjust dimensions when the input and transformed output do not have matching shapes.
Does a residual connection guarantee a better model?
No. It helps optimization in suitable architectures, but results still depend on the data, training and overall design.


