This is exactly the kind of resource that has been missing from the AI training ecosystem, and it deserves attention precisely because it refuses to pretend that GPU programming is magic. The creator is building a free, code-first series that walks from a naive GEMM kernel all the way to FlashAttention, and the framing is what makes it valuable: for most ML workloads, kernel performance is a data-movement problem before it is a compute problem. That single insight, grounded in the roofline model, is something too many practitioners learn only after burning weeks on the wrong optimizations. We have seen similar gaps in other tutorials. For instance, one recent project on Building a GPT-2 KV Cache to Measure Real AI Performance Gains showed how even straightforward mathematical savings can be misleading without a proper measurement framework. And a guide on Two Metrics That Fix What Every Quantization Guide Misses made the same point about threshold-dependent metrics hiding real bottlenecks. The pattern is clear: the field is full of high-level explanations and low-level code, but very few resources connect the two with a disciplined, profiler-in-hand approach.
What makes this series different is the structure. Each session follows the same loop: concept, live code, profile, and a check that the kernel actually got faster against a baseline. That last step is not a throwaway. In production, people regularly ship kernels that look clever but benchmark slower than a naive implementation because they optimized for the wrong bound. The series explicitly teaches how to read a kernel's position relative to the roofline ridge point, which tells you whether to optimize memory access or compute. FlashAttention is the worked example, and it is a smart choice because the gain is purely architectural: it computes exactly the same attention as the standard implementation, but tiles so the full N x N score matrix is never written to HBM. No approximation, no tricks, just a data-movement insight. The roadmap covers 44 sessions, from CPU parallelism and GPU architecture through to Hopper and Blackwell topics like TMA, WGMMA, and even AI-written kernels. That is an ambitious scope, and the creator is honest that later sessions may change as tooling evolves. That honesty matters.
The most useful feedback they are asking for is also the most telling: what do people who write production kernels usually get wrong that a course should cover? Our answer would be that most people underestimate how much time they should spend on profiling before writing any code. The roofline model gives you a ceiling; profiling tells you where you actually are. The series builds both into every session, and that is the right discipline. For readers who want to engage, the concrete takeaway is to run the first few sessions alongside your own work. Write a naive GEMM, profile it, and check whether your intuition about the bottleneck matches what the roofline model predicts. If it does not, that gap is where the real learning happens. The series is free, it is in progress, and the creator is not selling anything. That makes it a rare asset in a field where most deep-dives are locked behind paywalls or vendor documentation. Watch the intro video, run the first kernel, and send them the feedback they are asking for. That is how this kind of resource gets better.