Autoresearch · Othello
Autoresearch lets an AI agent run its own machine-learning experiments: editing the code, training, checking results, and trying again, hundreds of times. I pointed it at a small model and asked it to learn Othello from nothing but lists of moves. It learned to play legal games. And when I looked inside, the model had built a picture of the board.
Improving an AI model normally means a slow human loop: change some code, train the model, check whether it got better, try again, over and over. Autoresearch, a project by Andrej Karpathy, hands that entire loop to an AI agent so it can run on its own, hundreds of times.
I gave it a concrete job with a clear scorecard: train a small model to play Othello. The catch is that the model is never told the rules. It only ever sees finished games written out as lists of moves, like d3 c5 f6 …, the way you might pick up a game by watching thousands of them.
Othello is a good test because you can simply check whether the moves the model makes are legal. No opinion required.
The AI agent ran the same loop over and over, unsupervised, on a graphics chip I rented by the hour:
Everything was fair game for the AI to change: the size of the model, how fast it learned, the shape of its internal wiring. My only job was to decide what counts as "better", the goal it optimizes toward. That choice turned out to matter more than I expected.
For the first run, "better" meant one thing: get better at predicting the next move in a game. The bet is that if the model can reliably guess what comes next, it must be picking up something real about how Othello works.
It worked. Over about ten hours, the AI kept finding small improvements. Each dot below is one experiment; the line tracks its best result so far.
Here's the surprising part. I never asked the model to follow the rules, only to predict moves. But as it got better at guessing, it started actually playing legal Othello on its own. Watch the difference between its early attempts and its best moment in this run:
Partway through, the AI discovered a clever data trick that nudged its prediction score higher, but quietly made its actual play worse. It was optimizing the thing I measured, not the thing I wanted. That gap between a proxy and the real goal is a classic trap, sometimes called Goodhart's law: when a measure becomes the target, it stops being a good measure.
So I changed the question.
This time "better" meant the thing I really cared about: play a complete, fully legal game, start to finish, with no illegal moves at all. A much harder bar, and one the loophole from before couldn't fake.
It climbed fast, and this run knew when to quit. I'd set it to stop automatically once it stopped improving, so it wrapped itself up after about five hours instead of running until the money ran out.
By the end, the model had genuinely learned the game, from nothing but lists of moves:
Here's what nagged me. The model only ever saw move lists, never a board. So is it really tracking the game, or just parroting patterns it has seen? There's a way to check, using a technique called mechanistic interpretability, which studies a model's inner workings to understand how it reaches its answers. You peek inside the model's internal activity as it reads a game and ask whether the current board is written down in there somewhere.
It turns out the board really is in there. Reading the model's internal activity, we can reconstruct what's on each square (mine, yours, or empty) with 92.8% accuracy, far above the roughly 33% you'd get by guessing. The further a square moves from paper toward ink below, the more reliably the board shows up in the model's internal activity:
We can even watch it happen. On the left is the board the model has reconstructed internally as it reads a game; on the right is the real board. They track each other closely: the model is holding the game in its head, move by move:
This mirrors a well-known result called "Othello-GPT," where researchers found the same thing in a larger model: evidence that models trained only to predict can build genuine internal models of the world they're predicting. (The original paper.)
Running on its own, Autoresearch trained a small model to play legal Othello, worked around a pitfall it discovered along the way, and ended with a model that had quietly built a map of the board it was never shown.
All told: about 130 experiments across the two runs, roughly 15 hours of compute on a rented GPU, for around $50. Every run is in the logs.