Field notes · Adventures of Lolo · an interaction-only agent
An agent that gets pixels, a controller, and save states. No rules, no object names, no solutions, no demonstrations. Two days of experiments on a single room. This is the honest ledger: what failed, why each thing failed differently, and the four times the project proved itself wrong.
Adventures of Lolo is one of those core memory games for me. I would argue it is the best puzzle game on the NES after Tetris.
So this started from affection rather than from a benchmark. I wanted to know whether something could work these rooms out the way I once did, without being handed any of what I already know.
Adventures of Lolo is a 1989 NES puzzle game by HAL Laboratory, the studio that would later make Kirby. You are Lolo, a blue ball with a face, working through fifty single-screen rooms to rescue Lala from the Great Devil. Nothing scrolls. Nothing is timed. There are no reflexes involved.
Each room states its goal the same way. Collect every heart framer on the screen, which opens a treasure chest; take the chest, and the door out unlocks. What stands between you and the hearts is a fixed arrangement of blocks, water, and monsters, and a very small toolkit: you can push emerald framers one square at a time, and you can fire a magic shot that turns a monster into an egg. Eggs can be pushed, floated across water as a raft, and will hatch back into the monster if you leave them too long.
The monsters are part of the puzzle rather than an action challenge. Some sit inert until you cross their line of sight. Some patrol. Some shoot along a row or column the moment you step into it. Which ones are dangerous depends entirely on where everything else in the room is standing.
The game is deterministic and discrete, which makes it tractable to study. The difficulty is that it is a game about preparation with irreversible consequences. Pushing a block one square too far can make a room permanently unsolvable, and the game will not tell you. It keeps letting you play a room you can no longer win.
So the thing a player has to learn is not a reaction or a route. It is which rearrangements of the room open which futures, several moves before any of it pays off, with mistakes that are silent and permanent. That is the structure this project is trying to get an agent to discover on its own, without being told that a block is pushable, that an egg is a raft, or that a heart is worth having.
The agent may use four things: rendered RGB frames, controller actions and durations, opaque save/load capabilities, and the pixel-observed consequences of its own actions. It may not receive game rules, object names, level solutions, ROM memory, demonstrations, or solution videos. Internally an object is track-17; the data contracts have no field for push, enemy, or egg.
The target is stronger than finishing fifty rooms: solve held-out rooms with the learned parameters frozen, then attempt the sequel with them still frozen.
The preparation structure described above is what makes this hard to reward. It is invisible to every cheap objective: frame-level prediction, distance-to-goal, and raw novelty all report progress while the agent goes nowhere. Each of those was tried here, and each is in the failure list below.
The one passing result has since been re-confirmed on the fixed code path, in a preregistered pair of runs that both completed: the treatment collected the heart at decision 17 as predicted, the control never did, and neither lost a life. It is included here. Everything below is the assisted track, meaning the agent uses pixel detectors for the player and hearts as development scaffolding. The strict interaction-only claim, which is the actual research goal, has no object-centric evidence yet. That gap is the single most important caveat on this page.
One room, seen by the agent as pixels. The map below is derived from the frames themselves, using a per-cell modal colour over 80 recorded frames, not from any supplied level data.
The whole campaign turns on the difference between these.
The 7-cell and 24-cell sets are certified measurements: what a depth-12 search verifiably retained. The 67-cell set is what is actually reachable. Mistaking the first for the third cost this project two days.
A save state is a complete snapshot of the console's memory. You can restore one and the machine continues exactly as it was, every time. That gives this setting something almost no learning environment has: cheap, exact counterfactuals. A counterfactual is the answer to "what would have happened if I had done nothing instead?" Normally that question is unanswerable, because you only get to live one history. Here you can snapshot, press a button, snapshot again from the same instant, wait the same length of time doing nothing, and compare the two futures pixel by pixel.
That second run is the matched control: same starting point, same duration, only the action differs. It is the same logic as a control group in a drug trial, except it costs milliseconds and is exact rather than statistical. Every causal claim in this project rests on one, which is why the honest failures below are failures about what mattered rather than about what happened.
Two more consequences follow. The emulator is the oracle, the thing that settles questions: a learned model may propose an action, but only running it decides. And because branching is cheap, the agent explores by snapshotting promising positions into an archive and jumping back to them later rather than replaying from the start, the same family of trick behind Go-Explore.
Four pieces of method vocabulary appear throughout, and they are the reason the negative results here are worth anything.
All of it on one laptop. An Apple Silicon MacBook Pro, no cloud, no cluster, nothing rented.
The game runs in a headless emulator, meaning an emulator with no window and no speaker: it advances the console's state and hands back the raw picture as an array of pixels, as fast as the machine will go. The emulator core is Nestopia, driven through libretro, the same emulator plumbing RetroArch uses. It runs inside a small separate host process, which matters for the research rules rather than for speed: the ROM contents and the save-state bytes stay inside that process, and the agent's side of the boundary only ever receives RGB frames and opaque handles it cannot decode. The constraint is enforced by the architecture rather than by good intentions.
Speed is what makes the method possible. The host verifies roughly 586 branches per second and emulates around 9,400 frames per second, so a search that tries 25,000 different action sequences from one position costs about 43 seconds. A full experiment, 24 decisions with a wide search at each one, takes 30 to 60 minutes and produces about 85,000 telemetry events and 12,000 verified branches.
Every run writes a self-contained directory: an append-only event log, deduplicated frames, a manifest recording the digest of every input (ROM, emulator core, host binary, model checkpoints), and a decision-by-decision table. That manifest is why claims here can be checked rather than believed. It is also how the runs are known to be deterministic: give the system identical inputs and it produces a byte-for-byte identical run. Six separate control runs across four experiments produced the same 85,594 events and the same 24 decisions, which is what makes any difference in a treatment run attributable to the treatment.
One negative result about infrastructure, since it saved money: renting a cloud GPU was benchmarked and rejected. The bottleneck is a sequential, CPU-bound emulator, and the GPU did not accelerate it. A real-data training benchmark came back at 0.196 times local speed. The laptop won.
Each experiment replayed as its committed trajectory through the room. The markers are the moments the telemetry flagged: an archive restore, a deposit, the one collection.
The interesting part is not that things failed but that each failed for a different, nameable reason. Five separate mechanisms were built to make the planner prefer a prepared configuration.
Raw novelty and screen change. Useful for exploration, unreliable as progress. The agent spent most of its budget in title animation while its coverage metrics improved.
Straight-line distance to a visible goal. Accelerated easy routes, then stalled at every obstacle requiring preparation. Later hardened into a general rule: any consistent preference for target proximity fails, not merely distance rewards, because the excursions away from a target are what deposit the archive ladder that later progress climbs. Steering toward the goal destroyed reaching it.
Temporal precedence as causal credit. A button press was credited for a scene change 51 decisions later that a timer, not the agent, caused. Every causal claim since requires a duration-matched control from the same root.
Bigger search instead of better representation. One run rejected 1,756 changed-layout branches and accepted zero as reliable manipulation evidence, through 3,202 verified branches. The bottleneck was detection, not coverage.
A promotion gate asked whether a learned mask reproduced the incumbent's masks byte-for-byte. It passed on bits that turned out to be mask-irrelevant, then failed on a criterion no learned mask could satisfy, the incumbent's own defects included. Replacing it with a functional gate, refereed by detector-free ground truth rather than by the incumbent, revealed that the incumbent itself misses 6–12% of real manipulations.
These are the entries worth reading, because each was a conclusion already written down, already acted on, and then falsified by the project's own measurement.
After an experiment failed, the record said the failure had moved inside the planner's commit ladder, which had no tier referencing a target cell. Designing the fix falsified it: at the decisive decisions the planner returned early and never built a candidate set at all: branches_examined: 0. The ladder also already had such a tier. A new tier would have had zero opportunities across ten runs.
The closing mechanism won its contest and spent the win standing still.
The restore that was supposed to hold a valuable position selected the state the agent was already standing in, consumed the decision, and then removed that position from the archive.
The roadmap said the two remaining hearts lay outside the reachable region and therefore required a second manipulation. Three independent methods then put both hearts inside a 67-cell region reachable from the collected heart. An older run already held an emulator-verified 24-action branch that collects one of them with zero life loss. And the failing run's own search had climbed to five steps from the other heart at branch depth exactly 12, the configured ceiling, before a restore pulled it back.
Outside the certified envelope, therefore unreachable. A certified record is a lower bound
produced by one budget from one root. Depth 12 had been silently load-bearing across the entire campaign.
A six-event discrepancy between two runs was traced rather than dismissed as noise. The option search memoised matched no-op controls on (id(parent), duration), which is a memory address. Entry lifetime outlives key lifetime, so a reused address serves another node's control. It was reachable and firing, roughly one parent per search at depth four and beyond, and the corrupted value reaches the beam score. A wall collision was observed being scored as a measured local effect.
Impact is provably empty below depth four and unbounded at or above it, which is where every experiment on this page ran.
A monitoring check reported a run had died. Both of its signals were broken: the process pattern could not match the process, and the event count it called stalled was exactly on profile. The general rule that came out of it, that two readings sharing an author are one signal and not two, later caught the mirror-image error, a monitor matching its own search pattern and so calling a finished run alive.
The strongest positive result. Removing an object from the room was shown, under a certified configuration-hold predicate, to change reachable space from 7 cells to 24, including a cell bearing a heart. The comparison is honest in both directions: an earlier manipulation the project had celebrated and preserved for four generations was measured as strategically neutral. Value is a property of resulting configurations, not of manipulations.
A learned controllable-region tracker trained purely on counterfactual structure, with no player detector anywhere in the label path, now exceeds the assisted incumbent on stability, preservation of adjacent objects, and detection of in-place changes. It took six gate iterations, each isolating one mechanism, and it is still only promoted to shadow: it has never run in the planner.
This is the part that actually compounds. Preregistering the scoring rule before the run caught a discriminator that was quietly dying, a unit error that would have "fixed" a non-problem, a false-positive bound falsified before it cost a run, a lever credited with an effect it never had, and the correctness bug above. Most of the failures on this page were caught by measurement, not by luck.
Plain search at depth 24–36 did things the entire planning apparatus could not do at depth 12.
The immediate queue is short and ordered by what could invalidate what. The first item is now done: the one passing result has been re-confirmed on the fixed code path, so it no longer rests on a corrupted scoring input. Next is to re-measure reachability at greater depth against a depth-12 control. That is one cheap run that could retire a large amount of planned work.
Then a strategic question that this campaign forced into the open. The roadmap optimises a research claim: strict inputs, frozen parameters, transfer to unseen rooms and a sequel. The practical goal is beating the game. Those diverged, and nobody noticed for two days, because every experiment was scored against a gate rather than against room progress. Under the research claim the entire campaign scores zero, because all of it is assisted-track. Under the practical goal, the evidence now points at search budget rather than at planning.
There is a second-order irony worth recording. The rule "never raise depth after a failure" was earned honestly: it was written after searches got wider instead of smarter. Over-generalised past its precondition, it is exactly what stopped anyone re-measuring the budget once the representation finally existed.
Was Gate 4 measuring a planning capability, or measuring depth 12?
This experiment has been relatively costly, expending all my max-plan tokens on multiple services. There is a real opportunity cost that trades off against exploring other interests. There is a path forward here, and I may revisit it at a later time after thinking more about the research direction.
A short glossary of everything technical on this page, in the order it tends to matter.
| Term | What it means here |
|---|---|
| headless emulator | An emulator with no window or sound, advancing the console and returning raw pixels as fast as the CPU allows. |
| save state | A snapshot of the console's entire memory. Restoring one resumes play exactly, which is what makes exact counterfactuals possible. |
| branch | Restore a save state, take an action, see the result. The unit of work in every experiment here. |
| matched control | A second branch from the same instant that does nothing for the same duration. The comparison that turns "this happened" into "this action caused it". |
| archive | A store of promising positions the agent can jump back to instead of replaying from the beginning. |
| beam / depth | How many candidate action sequences are kept at each step, and how many steps ahead the search looks. Depth 12 turned out to be silently load-bearing. |
| telemetry | The event log a run writes. Everything asserted here was read back out of it rather than remembered. |
| determinism | Identical inputs produce a byte-identical run, so any difference between two runs is attributable rather than noise. |
| ground truth | An independent answer used to score a method, here derived from counterfactual structure rather than from the thing being tested. |
| falsification | Stating in advance the result that would prove you wrong, then looking for it. Four conclusions on this page died this way. |