Skip to content

Degeneration Problem in Normalization

Home / Concepts / Degeneration Problem in Normalization

Degeneration Problem in Normalization

Auto-generated stub. Edit this file to add more details.

A phenomenon where learnable non-linear normalization shape parameters collapse to linear limits during end-to-end gradient-based training.

Why It Matters

Identifies a critical failure mode in current adaptive normalization research where high-capacity backbones learn to ignore non-linear shaping.

Evidence

Training (δ,ε) jointly with the backbone via gradient descent reliably pushes them toward this linear limit within a few epochs — a phenomenon we name the degeneration problem

Metadata & Links