Paying Attention: Transformers from the Ground Up

13 episodes, about 25 minutes each

A 13-episode course that builds up the transformer, the engine inside modern artificial intelligence, starting from high school math, with every example drawn from robots and cameras. Two hosts, Maya and Leo, follow a robot arm named Sprocket as it learns to put the red block in the blue bowl. No visuals needed. Best heard in order.

Listen in Apple Podcasts

1. Copy this feed address: https://ih.github.io/paying-attention-podcast/feed.xml
2. In the Podcasts app, open Library, tap the ⋯ button, and choose Follow a Show by URL.
3. Paste the address and tap Follow. Episodes appear in order, starting with episode 1.

Episodes

1. 1. The Big Picture 26 min

   How does a robot arm turn camera pictures and a typed sentence into motion? A bird's-eye tour of the transformer: chopping inputs into tokens, description cards made of numbers, a stack of blocks where tokens consult each other and think privately, and a readout that becomes a movement. Plus why transformers took over, and the catches.

2. 2. Seeing in Numbers 24 min

   How a camera turns a scene into numbers, why any list of numbers can be pictured as a point or an arrow, and the dot product, the simple multiply-and-add calculation that measures how well two lists match. We build a redness detector and an edge detector by hand.

3. 3. Chopping the World into Tokens 23 min

   How sentences become word pieces, pictures become tiles, and robot movements become clicks on a dial, and how every token gets a learned description card, called an embedding. Including how CLIP put pictures and captions into the same space.

4. 4. Building a Brain Cell 26 min

   The artificial neuron, built by hand as a grasp detector for Sprocket's fingers. Why one neuron can't detect "in the middle," why the bend matters, how layers and matrices work, why graphics chips are so good at this, and what vision networks learn layer by layer.

5. 5. Learning from Mistakes 24 min

   How machines learn from examples: recording demonstrations, measuring mistakes with a loss, walking downhill in the fog with gradient descent, and passing blame backward with backpropagation, all worked through with small numbers. Plus overfitting and snowballing errors.

6. 6. Why Attention? 25 min

   Why context matters so much, how convolutional and recurrent networks struggled with it, the 2014 and 2017 breakthroughs, and the core idea of attention: every token asks a question, wears a label, and offers a package of information.

7. 7. Attention, Step by Step 25 min

   The complete attention recipe, done with numbers small enough to do in your head: dot-product scores, dividing by a square root, the softmax, blending values, masks that stop peeking, and cross-attention.

8. 8. Many Heads and a Sense of Place 24 min

   Why transformers run many attention heads side by side, what researchers find those heads doing, why attention is blind to position, and four clever ways to tell the model where and when everything is.

9. 9. Inside the Block 26 min

   A complete transformer block: the feed-forward rulebook where much of the knowledge lives, mixture of experts, residual connections that add instead of replace, layer normalization as auto-exposure, and how a tile's card changes as it climbs the stack.

10. 10. From Thoughts to Actions 24 min

    How the final cards become movements: menus and softmax, action tokens versus direct numbers, the averaging trap that steers straight into the vase, temperature, noise-refining action heads, action chunks, and the tricks that make robot brains fast enough.

11. 11. Training a Robot Brain 25 min

    How robot brains are trained in practice: learning to see from internet pictures, the surprise score measured in bits, scaling laws, where robot data comes from, fine-tuning on demonstrations, small add-on dials, and learning from trial and error.

12. 12. The Family Tree 25 min

    Encoder-only, decoder-only, and encoder-decoder transformers, and the famous relatives in vision and robotics: the Vision Transformer, Swin, the Detection Transformer, Segment Anything, CLIP, RT-2, ACT, pi zero, the Decision Transformer, Gato, and world models.

13. 13. One Moment in the Life of a Robot Brain 25 min

    The finale: one decision followed through the whole machine with real numbers, from camera snapshot to motor commands, including recovering from a dropped block. Then the honest limits, safety, what's coming next, and the whole series in a nutshell.