What is a residual connection?

Definition

Instead of requiring a block to learn an entire mapping directly, the architecture provides a shortcut alongside the learned transformation. This supports residual learning and can make deep networks easier to optimize.

For example, a block can preserve the incoming representation while adding a useful correction. Addition requires compatible shapes or a suitable projection; the shortcut alone does not guarantee that training will succeed.

ELI5

A block can keep the information it received and add a learned adjustment.

For example, think of revising a draft by adding a correction instead of rewriting everything. The original path remains available alongside the change.

Is the shortcut always an unchanged copy?

No. A projection can adjust dimensions when the input and transformed output do not have matching shapes.

Does a residual connection guarantee a better model?

No. It helps optimization in suitable architectures, but results still depend on the data, training and overall design.

  1. Pierce Freeman and Richard Diehl Martinez in blue and white tops against black, one touching his chin, alongside the blue and white headline "DEEPSEEK'S NEW BLUEPRINT".
  2. Pierce Freeman and Richard Diehl Martinez in blue and white tops against black, alongside the blue and white headline "WHY AGENTS CHEAT".
Definition card for residual connection: What is a residual connection?

A residual connection adds a neural-network block's input to its transformed output, allowing the block to learn a change relative to that input.