AI Pixel Cat
A pixel-art cat that lives on a $30 development board, has five gauges you can fill over a five-minute care session, hides in the park when you shake him, and, after about a week of watching, starts meowing at the hour you usually turn up.
It started with someone else’s fluid simulation
The reason any of this exists
In early August I came across a post by Johannes Tscharn showing a fluid simulation sloshing around on a tiny round-cornered display, tilting with the board in his hand. “This replaces scrolling for me now,” he wrote. It had three quarters of a million views, and I understood why immediately: there is something about a physical object with a live little world inside it that a browser tab has never managed.
He posted the board he used, and a day later the repo. He also signed off with a line that turned out to be the whole prompt for this project: “Will do something that leverages mic + speaker next…”
I bought the same board and built a Tamagotchi-inspired AI pixel cat.
The device
Waveshare ESP32-S3-Touch-AMOLED-1.8 · $29.99
The appeal of this particular board is that essentially every sense a virtual pet could want is already soldered on. It is a complete creature for thirty dollars, battery included.
| Hardware | Part | What it became |
|---|---|---|
| Display | 1.8″ AMOLED, 368×448, CO5300 over QSPI | The park (rendered sideways as 448×368) |
| Touch | CST820 capacitive | Petting, tapping, swiping to search |
| IMU | QMI8658 6-axis | Tilt makes him walk; a shake scares him |
| Audio | ES8311 codec, mic + speaker | Purring, hissing, meowing, and listening |
| SoC | ESP32-S3R8, 8 MB PSRAM, 16 MB flash | Two 1.75 MB worlds resident in PSRAM |
| RTC | PCF85063 | Time of day, which sets the sky |
| Power | AXP2101 PMU, 3.7 V lithium | Battery gauge, and sleeping to save it |
| Storage | TF card slot | Event log, boot log, captured warnings |
What I wanted to build
The requirements, before any code
Inspired by a Tamagotchi, but only up to a point. A Tamagotchi begs, and then it dies, and the threat of death is what makes you press the buttons. A cat is a different proposition entirely: a cat that has been neglected does not die, it simply stops being especially interested in you. That asymmetry became the design.
- No death, no sickness, no fail statesNeglect makes him aloof, attention makes him yours, and everything is always recoverable. The worst outcome is a cat who wanders off and ignores your taps.
- Every gauge aspires to be fullAn early version had a “hunger” bar, and a full hunger bar reads as a starving cat. Renaming it Food fixed the whole mental model: five bars, all of which you want topped up.
- A five-minute care session, not a five-day choreDo everything once: a feed, a play, a petting, a walk, and a nap, and every gauge fills. Walk away and they all drain within minutes. His sleep ends the episode and resets the rest.
- He has to use his sensesThe mic, the speaker, the accelerometer and the touchscreen all had to do something a cat would plausibly do, rather than sit unused on the bill of materials.
- Everything runs on the deviceNo cloud, no account, no inference API. The only thing that ever touches the network is an NTP time sync, because the onboard clock runs at half speed and cannot be trusted.
- He should learn when I actually show upAnd then get my attention at those times, on his own initiative, which is the part that makes him feel less like a screensaver and more like something with an opinion.
The cat
Ten cycles, 16 pixels wide
The cat himself is not mine. He began as Elthen’s 2D Pixel Art Cat Sprites, a sprite sheet I bought and then modified, recolouring him so he grades with each sky and giving him the green eyes that glow at night.
From there every pose is transcribed into ASCII art, one character per pixel, with w for the white body, s for grey shading, and k for an eye, then compiled into the firmware. Moving the frames into plain text was the best decision in the pipeline: they diff cleanly in git, they are editable in any editor, and a fifty-line script renders a whole cycle to a contact sheet, so a walk can be judged as a walk rather than as eight separate drawings.
A rule I gave myself
Some animations are reserved. He only washes a paw after eating, the way a real cat does, and batting and pouncing only exist inside a play session. Making an animation rare is most of what makes it feel like behaviour instead of decoration.
The park
Five skies, and an embarrassing lesson about compression
The world is a looping park, 2,496 pixels wide, in five variants that follow the clock: day, dawn, dusk, twilight, and night. The cat and the effects ride the same colour grade as the sky, so at night he goes blue and his eyes glow green.
Day
Dawn
Dusk
Twilight
NightFive variants at 2,496×368 pixels in 16-bit colour is 9.19 MB of art, against 16 MB of flash that also has to hold the firmware. So I built an SD-card streaming path. Then a WiFi fetch. Then a whole WiFi access point with an upload page, so new art could be pushed to the board over the air.
The lesson
I built three delivery mechanisms for 9.19 MB of art without once checking whether it compressed. It is 8-pixel blocky art with enormous flat regions: it DEFLATEs to 129,711 bytes. That is 70.8:1. All five variants now ride along inside the firmware and inflate into PSRAM through the ESP32’s ROM, in a fraction of a second, and all that delivery machinery was deleted. Measure the data before you build the pipeline.
Reading the cat at a glance
Five icons that are also the gauges
There is no status bar and no numbers. Each icon is drawn twice, once in quiet grey and once in colour, and the colour fills bottom-up, one pixel row at a time, as the gauge rises. The icon is the reading. Tapping the fish drops a bowl; tapping the yarn ball starts a play session; tapping any of the others opens a menu with the words, a check or cross for whether that need was met today, and a day streak.
| Gauge | Fills from | Drains in |
|---|---|---|
| Play | Yarn-ball sessions: batting and pouncing | 5 min |
| Food | Eating; one mouthful is one gauge row | 6 min |
| Love | Petting until he purrs, play, making peace | 5 min |
| Exercise | Walking and dashing only, never panic or pounces | 5 min |
| Sleep | Fills while he naps; about 75 seconds does it | 6 min |
A filling exercise bar is not a demand; it is a pleasantly worn-out cat, who paces slower, loafs longer and naps earlier. And when the sleep gauge completes, everything else resets to zero: his nap ends the episode, and you both start fresh.
Scaring him, and making up
The part that gives him dignity
Shake the board and he arches, hisses, and the instant the hiss finishes he bolts, chaining the leap cycle at speed and flying off the edge of the screen in under a second. He hides somewhere in the looping world, and it costs you affection: the heart empties one row at a time, each row with a beep a little lower than the last.
To find him you swipe to pan the camera through the park. Soft footfalls tell you the search is getting warm. But he is not waiting for you; whenever the camera has him in view he keeps walking away, and you have to tap him mid-stride.
Reconciliation is purr-gated
Tapping him brings him out, but wary; the fear level persists. To actually make peace you have to pet him until he genuinely purrs. Only that resets the escalation and wins back affection. Scare him repeatedly without making up and he hides further away each time, up to half the world from wherever you happen to be looking. A purr cannot be faked. It has to be earned.
He learns when you show up
Two small models, both on-device
This is the part I actually wanted to build, and it is deliberately not a neural network. It is two classical models small enough to live in a few hundred bytes of flash, learning from the first day.
1. The schedule predictor
The week is divided into 96 half-hour buckets: 48 slots in a day, doubled because weekends do not look like weekdays. Each bucket holds one number: the probability that you turn up in it.
When a half-hour slot closes, its bucket is updated by whether you actually appeared. That is the entire learning rule, an exponential moving average:
The elegance is in what it does not need. There is no history to store and nothing to retrain: the single number is the summary of every visit that slot has ever seen, with older evidence decaying automatically. After fourteen contradicting observations only about a tenth of the old belief survives, which is what lets a changed routine overwrite an old one instead of averaging with it forever.
It also costs nothing to consult. Asking “does this half-hour deserve a wake?” is one array index and one comparison, which is why he can afford to ask roughly every twenty seconds all night.
2. The enticement bandit
Knowing when is only half of it; he also has to decide what to do about it. When a wake fires he performs an opening act, such as jumping about, loud purring, pawing the glass, pacing, or meowing, and then watches whether it worked.
This is the classic multi-armed bandit problem: five acts, unknown payoffs, and every performance is both a chance to score and a chance to learn. He keeps a value per act per time of day, with four periods × five arms and twenty numbers total, updated with the same one-line rule as the schedule:
Choosing is epsilon-greedy: 20% of the time he tries a random act, and the rest of the time he plays his current best. That 20% is what stops him getting stuck; an act that got unlucky twice early on still gets another hearing. The values also start optimistic, at 0.60, so every act looks promising until tried and none is written off unheard.
The proactive wake
While light-sleeping on battery he checks the clock against the schedule model roughly every twenty seconds. Entering a confident bucket wakes the device fully: screen on, cat awake, a procedural meow, then his learned enticement. He gets a five-minute audition window. Interaction counts as a hit; a timeout is a miss, logged, and straight back to sleep.
Misses cost that bucket confidence, and a global precision target raises his threshold automatically when he over-predicts. When he is wrong too often he gets choosier, not louder.
A predictor that is merely enthusiastic is worse than useless on a device that meows. So a second loop sits above the first, watching his hit rate and moving the bar he has to clear:
Every wake then has to pass all five of these, in order: mature enough, charged enough, under the daily cap, not already fired this slot, and confident enough:
How all of this fits on a $30 board
The honest answer is that it fits because it is small on purpose. There is no inference framework, no matrix multiply, no training set retained on disk, and nothing that needs a network. Both models together are 512 bytes: one static struct, no heap allocation, written to flash as a single blob.
For scale: a single park variant sitting in PSRAM is 1.75 MB, about 3,600 times larger than everything he has ever learned about you. The cat is enormously more expensive to draw than to think.
Why classical beats neural here
A neural network would need training data kept somewhere, a forward pass costing real milliseconds, and floating-point work every time it was consulted. This needs a subtract, a multiply and an add per observation, and a single comparison to make a decision. That is the difference between a model you run occasionally and one you can afford to consult every twenty seconds, for weeks, on a battery, even while the chip is asleep between checks. Small does not mean crude here; it means the learning can be continuous, which is the property that actually matters for something pretending to know your habits.
There are no quiet hours, by design
He only predicts the hours you taught him. But if you taught him 3 am, he will meow at 3 am. Do not keep this device in your bedroom. It is a cat.
The code
C, ESP-IDF 5.5, about 1.17 MB of firmware
Everything is in the repository: the sprite sources, the world generator, the two models, a host test harness that runs the gameplay logic on a laptop, and a preview tool that renders any game state straight to a PNG. That is how every screenshot on this page was made, without a flash cycle.