Classical Supervised ML

Fill in the classical classification concepts around margins, kernels, asymmetric costs, and imbalance-aware objectives.

Estimated time: ~75 min

Study this path with flashcards

5 cards

Step 1
Hinge Loss
Hinge loss penalizes examples that are misclassified or that lie too close to the decision boundary. It is the convex margin-based loss underlying soft-margin SVMs and emphasizes confident separation rather than calibrated probabilities.
Step 2
Support Vector Machine (SVM)
A support vector machine finds the decision boundary that maximizes the margin between classes, depending only on the support vectors nearest the boundary. With kernels, SVMs can model nonlinear separators while retaining a convex optimization objective.
Step 3
Kernel Methods
Kernel methods turn linear algorithms into nonlinear ones by replacing inner products with a kernel function that implicitly measures similarity in a higher-dimensional feature space. This is the core trick behind SVMs, kernel ridge regression, and Gaussian processes.
Step 4
Cost-Sensitive Learning
Cost-sensitive learning assigns different penalties to different kinds of mistakes instead of treating every error equally. It is the right framework when the real objective is to minimize downstream harm or utility loss rather than raw misclassification rate.
Step 5
Class Imbalance and Reweighting
Class imbalance means some labels are much rarer than others, so an unweighted objective can be dominated by the majority class. Reweighting changes the loss or sampling scheme so rare classes exert more influence during training.