A residual connection lets a neural-network layer add a transformation to an existing representation instead of replacing it completely. A gated residual connection adds a learned control that regulates the size or content of that update. The model can preserve an earlier signal when a new transformation would be unhelpful.
Gating can improve information flow through deep networks and reduce repeated overwriting across layers. It also gives the model another mechanism to decide when to retain or modify context. The exact gate design varies by architecture, and its benefits must be measured against the added parameters and computation.
Acronyms and aliases
gated residual branch variant
Related terms
Frequently asked questions
What does the gate in a residual connection do?
It learns how much of a layer's new transformation should be added to the representation that is already flowing through the network.
Why can gated residual connections preserve information?
The gate can reduce an update that would overwrite a useful earlier signal, allowing that information to remain available to later layers.
