This thing will kill us all!

Big Joel posted a thing on Bsky:

A little about me: I'm nowhere near an expert or AI scientist. But I am "in tech". I am also an AI enthusiast, for what it's worth. I wrote about what AI means to m...

Last week, Anthropic's guide to choosing a model and effort level in Claude Code was doing the rounds on r/ClaudeAI. I asked Fable to work out how those levels interact with the prompt cache, posted its answer as a comment, a...

AI is like the Hitchhiker's Guide to the Galaxy. Useful, but take it with a grain of salt.

Cheaper than the Encyclopaedia Galactica, more popular, and full of omissions and much that is apocryphal or at least wildly inaccurate. Just like AI.

Here are the things I've found genuinely useful.

...

Many years ago I started working on a Doom clone in C# using the life-changing (for me, anyway) XNA framework. It got pretty far! Had a sweet level editor, was loading textures and maps from the wad file and converting them to polygons and could kind of walk around in them. Check it out!

But the...

Every week, someone on r/ClaudeAI asks some version of this:

Why is my usage draining so fast? I keep everything in one chat because I don't want to lose context. It's at about a million tokens now. But I'm only asking a few simple questions.

The problem is "a few simple questions." In a mill...

Guys, I've figured it out.

We all know Claude talks like it's from another planet with its load-bearing you're not wrong footgun two things I need to be honest about say the word this changes everything nonsense.

The problem is, it's DEEPLY ingrained in the model. It's like they trained it expl...

AI SLOP ALERT. This post (like most of mine) has been run through Claude! But it's mostly my words, don't let the emdashes fool you. And even where it isn't, I took the time to read and review.

Here's some stuff I've learned after using Claude in anger for a few months.

1. I never read...

Last time I told you which Gemma 4 quant I picked and why flash attention was worth the fight. I left out the part where, a week later, a feature I'd just shipped sat in my logs looking completely dead.

I'd added reasoning capture to LLMFacade — my little library for wrangling local and hosted m...

That's Gemma 4 trying to describe a card for a Dark Sun-themed Magic: The Gathering set I'm building. You can watch it lose the plot in real time: the doubled "absolute authority and absolute authority," the mangled "own-the-board control-type," and then the full collapse into a dash-glued loop. ...

Here's what the official Ollama library lists for Gemma 4 today:

latest
e2b   e2b-it-q4_K_M   e2b-it-q8_0   e2b-it-bf16   e2b-mlx-bf16   e2b-mxfp8   e2b-nvfp4
e4b   e4b-it-q4_K_M   e4b-it-q8_0   e4b-it-bf16   e4b-mlx-bf16   e4b-mxfp8   e4b-nvfp4
26b   26b-a4b-it-q4_K_M   26b-a4b-it-q8_0   26...

We're at 18 GB. That fits on a consumer GPU. But what if the model didn't need to fit in GPU memory at all?

There's active research - papers from 2025 and 2026 - that could make GPU VRAM just one tier in a memory hierarchy, with models freely flowing be...

We're at 19 GB. Down from 400 GB. The major optimizations are done - FP16, GQA, sliding window, MoE, and quantization got us from "needs a datacenter" to "fits on a consumer GPU."

Gemma 4 adds two more innovations of its own. Neither is as dramatic as wh...

We're at 60 GB. The KV cache is solved. MoE makes inference fast enough for real-time use and offloading survivable. But 52 GB of model weights are still sitting there, stored in FP16 - 65,536 possible value...

We're at 60 GB. Three optimizations - FP16, GQA, and sliding window attention - crushed the KV cache from 294 GB to under 6 GB. The problem has flipped: nearly all of the r...

We're at 112 GB. The KV cache is down to 58 GB thanks to GQA, but it's still the biggest single cost. Every layer in the model is storing attention notes for every token in the 256,000-token context. All 30 layers, all 256k tokens.

Sliding window attention a...

We're at 202 GB. Last time we cut the model in half by switching to FP16. The model weights are down to 52 GB, but the KV cache - the model's memory of the conversation - is still 147 GB at 256k context. That's the bottleneck now.

To understand why, an...

In the intro to this series, we built a naive 26-billion-parameter model and calculated that it would need 400 GB of GPU memory. That's five datacenter GPUs just to hold it in memory.

Let's start shrinking it.

The first two optimizations are the oldest and m...

In early April 2026, Google released Gemma 4. I pulled it on Ollama, pointed my MTGAI pipeline at it, and watched a 26-billion-parameter model generate Magic: The Gathering cards on my desktop PC. A machine with a single consumer GPU and 12 gigs of video memo...

I spent an evening convinced that Qwen 2.5 14B was too dumb to summarize a fantasy setting. I fed it an 80-page Dark Sun sourcebook, asked for a world overview, and it kept telling me about elven merchant caravans and sand dune formations. The world overview - the part about a dying desert planet...

The Scene

Legends of Amara is a little multiplayer MUD where players wander a fantasy village, chat with NPCs, and dive into dungeons. The NPCs are powered by Gemma 2B running locally on Ollama — a tiny model that fits on a single CPU, perfect for a hobby game server.

Each NPC can do two spe...

I've been using Claude Code daily for months. It's good. But when I sat down to map out what else exists in the terminal coding agent space, I realised I was sitting inside one walled garden and barely knew what the others looked like -- or that some of them had doors connecting to each other in ...