A family of techniques that constrain or modify the learning process to reduce overfitting and improve generalization, typically by penalizing model complexity or injecting noise during training.

Geometric intuition for regularization. The sharp corners of the L1 penalty (left) cause intersections on the axes, driving specific parameters exactly to zero (sparsity). L2 (right) shrinks all parameters uniformly.

Regularization addresses the fundamental problem that a model with sufficient capacity can memorize the training set perfectly while failing on unseen data. All regularization methods work by restricting the effective hypothesis space, either explicitly through the objective function or implicitly through the training procedure.