Blog
Writing on ML systems, GPU kernels, and LLMs.
- September 2, 2026
Inside the CDNA5 Tensor Data Mover
CDNA5 adds a small DMA engine that copies tiles of a tensor from global memory straight into LDS, without the shader core computing a single address. What it is, how you drive it, how the descriptor describes a copy, and how the counter lets the copy overlap the math. Checked against an MI450.
GPUAMDCDNA5kernelsTDMLDSGEMM - August 9, 2026
Beating torch.relu with FlyDSL: A Hands-On Guide to Bandwidth-Bound Kernels
Simon Boehm's CUDA matmul post, but in one dimension. Writing the simplest GPU kernel there is in AMD's FlyDSL, then taking it from 2.9 to 7.2 TB/s on an MI350X — 1.04x torch.relu at the tuned shape, 1.28x at its best. Five versions with the code, what each optimization was actually worth, the two that measured negative, and the testing discipline a layout DSL asks for.
GPUAMDCDNA4kernelsFlyDSLMLIRbandwidth - June 8, 2026
MXFP8: Microscale Floating Point 8 — How Block-Level Scaling Makes 8-Bit Training Work
A from-first-principles look at the MXFP8 datatype: why regular FP8 isn't enough, how per-block scaling stretches 8-bit dynamic range by 2.5×, the hardware plumbing on CDNA4 and Hopper, and the block-size theory behind the design.
GPUAMDFP8MXFP8quantizationLLMCDNA4 - May 31, 2026
Occupancy Math on the AMD MI355X (CDNA4): A From-First-Principles Guide
A from-first-principles guide to wavefront occupancy on AMD's MI355X (CDNA4): the hardware resource budget, the four limiters that cap it, worked MXFP8 GEMM examples, and why peak throughput often lives at low occupancy.
GPUAMDCDNA4kernelsoccupancy