ResNet: why 152 layers can beat 20
Before 2015, adding layers beyond a point hurt accuracy—even on the training set. Not overfitting: the network simply stopped learning. This was called the degradation problem. He et al. fixed it with one idea: if extra layers add nothing useful, let them learn to predict zero. Then the block output equals the input (identity). A skip connection makes that trivial—and the gradient always has a direct highway.