A model with billions of parameters, running inside a tab, on the GPU of whoever opens the page, with no server behind it. That was the idea. And the most instructive part wasn't the day the engine started talking — it was five bugs that kept every test green.
The engine is written from scratch on WebGPU and WGSL — the browser's GPU API and its shader language — with no dependencies. Not by choice: the available alternative was unusable, because its license stopped applying above a certain revenue threshold.
Writing an entire runtime over one clause sounds like overkill until it's your turn. The interesting part is what comes next, when every piece has to be solved by hand and the model refuses to cooperate.
The weights ship as a single file in GGUF format, which opens with a header and a tensor index: what's inside, how big, in what format, and at what offset in the file.
The loader reads only that opening slice, with range requests — an HTTP request that asks for specific bytes instead of the whole file — and from that alone it knows which architecture it's looking at, how many layers there are, and what format each piece uses. It's a tiny fraction of the total.
It looks like plumbing, and it changes the whole startup: you can know what you're about to load before you've downloaded anything meaningful.
The model is quantized down to a little over a bit and a half per weight, but not uniformly: several formats coexist inside the same file, because the early layers carry more precision than the rest. A sensible decision by whoever quantized it, and a nuisance for whoever has to run it.
Each format needs its own GPU program to undo it. There's no shortcut: one shader per format, written by hand.
That's where the decision that saved the most time later got made. Every dequantizer was validated with an exact round-trip — quantize, dequantize, compare — against the reference implementation in Python. Not "close enough within a tolerance": bit-for-bit identical.
A dequantizer that's almost right doesn't throw an error. It gives you a slightly dumber model, and that doesn't show up anywhere.
This model is a hybrid. Most of its blocks aren't attention — they're recurrent, state-space: instead of looking back over the whole context at every step, they carry a state forward. The engine needs two distinct paths internally, and that has a rather beautiful consequence that deserves its own article.
What matters here is that the architecture is read out of the file. Assuming it is the quickest way to get it wrong.
Every tensor gets its own GPU buffer. And that's where a hard browser limit appears: the maximum size you can have bound to a shader at once.
On desktop that ceiling is generous and you never find out it exists. On mobile you do: the largest tensor doesn't fit, and it has to be split.
It's the kind of limit you don't discover by reading the spec — you discover it when somebody opens the page on their phone.
The file shipped with an extra head for predicting several tokens at once, which this engine doesn't run. It was just sitting there, taking up space.
It's detected from the metadata, not from the tensor name — hardcoding names is how you break on the next model — and skipped at load time. A decent chunk of the file that no longer gets uploaded to the GPU every session, for free.
And now the good part. Five bugs turned up that no test caught, because no test was failing:
None of them threw an exception. None of them painted anything red. All of them returned plausible text, which is exactly what makes them dangerous: a silently degraded model keeps on talking, and you're sitting there perfectly happy.
The lesson is short: every piece gets validated against an external reference, written by other people, never against itself.
In practice that meant three comparisons, not one. The full forward pass is checked against the reference implementation in C++ and agrees within tolerance. The tokenizer has to return the same ids, case by case, not similar ones. And the batched generation loop has to match the token-by-token one exactly, which is the cheap way to catch state getting dirty along the way.
Three boring tests. They're exactly the ones that separate "it works" from "it looks like it works".
Part of a series on building a browser-native inference engine.