Study a topic in one sentence, open the derivation only when you need it, and return later with spaced repetition.
Entry points
Guided sequence
Best for learners who want ordered progression from foundations to advanced topics.
Thematic browsing
Best for readers who already know what neighborhood they want: optimization, attention, systems, or RL.
Spaced repetition
Best for retrieval practice after reading. The study loop is one click away.
Fresh material
Best for repeat visitors who want new additions, recent edits, or timely topics without scrolling a path list.
Plan your study
Best for readers who want to set a concrete goal and build a sustainable routine before diving in.
Beyond the wiki
Best for finding complementary websites, books, courses, and videos to deepen study outside the wiki.
Curriculum
Learning path
206 topicsBuild a solid mathematical and conceptual grounding. Covers linear models, error decomposition, and essential optimization.
Summary-first · Dig deeper · Study mode
Learning path
285 topicsTrace the evolution up to modern landmarks. Covers CNNs, initialization, and Memory Networks.
Summary-first · Dig deeper · Study mode
Learning path
339 topicsMap the trajectory from word embeddings to modern generative pre-trained transformers.
Summary-first · Dig deeper · Study mode
Learning path
10 topicsEnsuring models behave predictably. RLHF, DPO, reward modeling, and the frontiers of mechanistic interpretability.
Summary-first · Dig deeper · Study mode
Browse by idea family
Mathematical foundations
Optimization & training
Attention & transformers
RL & alignment
Systems & scaling
Vision & generative
Recently updated
deep-learning
Equivariant neural networks bake a symmetry group action into the architecture so that \( f(g \cdot x) = \rho(g) \cdot f(x) \) holds exactly by construction, achieved by constraining each layer's weights to the operators that commute with the group action. The dominant examples are CNNs (translation), Deep Sets and GNNs (permutation), G-CNNs (discrete rotations), and SE(3)-equivariant networks like EGNN, NequIP, and AlphaFold's IPA (3D rigid motions for molecules and proteins).
llm
Compressed Sparse Attention (CSA) is a long-context attention scheme that first compresses the KV cache into block summaries and then performs sparse attention only over the top-k relevant compressed blocks. An added sliding-window branch preserves exact local dependencies, so CSA cuts both KV memory and long-context attention compute without collapsing into a purely local window.
deep-learning
Offline reinforcement learning learns a policy from a fixed logged dataset without further interaction with the environment. Its central difficulty is distribution shift: Bellman backups evaluate actions that are poorly supported by the data, so modern methods either constrain the learned policy to stay near the behavior data or pessimistically down-value unsupported actions.
deep-learning
Model-based reinforcement learning learns an explicit or latent model of environment dynamics and uses that model for planning, imagination rollouts, or policy optimization. Its main advantage is sample efficiency, while its main failure mode is model bias: the policy can exploit errors in the learned simulator unless planning and training control compounding prediction error.
deep-learning
Decision Transformers cast offline reinforcement learning as conditional sequence modeling: a Transformer predicts the next action from past returns-to-go, states, and actions. This avoids explicit Bellman backups and instead treats policy learning like autoregressive imitation conditioned on the desired future return.
deep-learning
Multi-agent reinforcement learning studies environments where several learning agents interact simultaneously, making each agent's dynamics depend on the evolving policies of the others. The main challenges are non-stationarity, coordination, and credit assignment, which is why centralized training with decentralized execution is a common modern design.