Notes from the lab.
Field notes from Melbourne on evals, inference, and the product we are shipping. For builders, not investors.
Building the machine a model actually wants
A model, once you stop thinking of it as math and start thinking of it as a machine, is a dataflow. The weights stream in once and then sit still, the key and value for every token pile up and get read again by every token that follows, and the activations move from a projection into attention into a mixture of experts and back into the residual while a small sum runs between the GPUs so each one ends up holding the whole answer. Serving a model is really just moving those tensors along a path,
Writing our own inference engine in Rust on the AMD MI355X
On the small all-reduce that tensor-parallel inference runs for every token, our from-scratch Rust inference engine comes out about 1.3× faster than AMD's RCCL at its best, sitting on the physical floor of the MI355X.
Same Dictionary, Different Geometry
Recent work shows Muon representations are more robust and transferable. We ask what that means anatomically: at 124M scale, firing-pattern matching says the sparse features are the same; Muon recovers the AdamW feature set up to seed noise, packaged into a hotter, sparser, more crowded, and more seed-reproducible residual-stream geometry.