Blog posts
RSSDeep dives on LLMs, AI agents, and ML systems, and the engineering underneath the tools.
- Attention Is O(n²): FlashAttention vs Linear Attention Standard attention on a 32K-token sequence allocates 2,199 GB of score-matrix memory across a 32-layer model — 27× the capacity of an A100 80GB.
- LLM Under the Hood — Part 5: The Transformer Encoder Build a complete Transformer Encoder Layer in PyTorch with residuals, LayerNorm, and FFN. Real print(model) output of an 8,544-parameter encoder.
- Positional Encoding: From Sinusoidal to RoPE to ALiBi Transformers are permutation-invariant by design: without positional information, the model treats 'the cat sat on the mat' identically to 'the mat sat on the cat.' Positional encoding injects order…
- LLM Under the Hood — Part 4: Multi-Head Attention Run attention in parallel across multiple heads, then inject word order with sine/cosine positional encoding. Real PE matrix visualized in PyTorch.
- Entropy, Cross-Entropy, and KL Divergence in LLM Training Each code block in this article is self-contained and can be run independently — that is why a few helper functions (softmax, kl_divergence, etc.) are re-defined across snippets.
- LLM Under the Hood — Part 3: The Attention Mechanism Build scaled dot-product attention from scratch in PyTorch. See real attention weights from a forward pass. The mechanism that powers every modern LLM.
- Backpropagation Without the Magic: A First-Principles Derivation Every ML engineer uses backpropagation daily. Most treat it as a framework primitive — loss.backward() runs and gradients appear.
- LLM Under the Hood — Part 2: From RNNs to LSTMs Why vanilla RNNs fail on long sequences and how LSTM gates fix it. Real PyTorch training and BPTT gradient measurements. Part 2 of LLM Under the Hood.
- Why Transformers Work: Attention as Learned Soft Retrieval Every practitioner working with transformers has encountered the attention mechanism described the same way: 'it lets tokens attend to each other.' That description is accurate and useless.
- LLM Under the Hood — Part 1: Introduction & Word Embeddings A 10-part series rebuilding modern LLMs in PyTorch from scratch. Part 1 starts with word embeddings and a real CBOW model trained on a tiny corpus.
- Feature Store Architecture: Fix Training-Serving Skew A recommendation model that performs well offline but degrades in production is one of the most common and most expensive problems in machine learning.
- Async DAGs vs Fibers: Stackless and Stackful Concurrency in C++ A future, a promise, a reactive stream, a sender — they're all the same async DAG: a graph of continuations that fires as results resolve. Fibers express the same graph as sequential code with a stack. Here's the architecture of each, down to the coroutine frame and the register swap, and how to choose.
- Multi-Agent Fan-Out: When Parallelism Bites Back Scatter-gather is one of the most seductive patterns in distributed systems: split a hard problem into N pieces, run them in parallel, collect the results.