From 10p in the slot to a robot called Chip
Hiya, I'm Jonny. I run a little shop on Dickson Road in Blackpool, and when it's quiet, I tinker with AI.
As a youngster I practically lived in the amusement arcades. I grew up with them: the noise, the lights, a pocket full of 10ps. As an adult, technology's always fascinated me, and so has AI.
In 2015 I read about a computer that taught itself to play Atari games from nothing but the picture and the score, and I had to have a go. Back then that meant Linux on a bootable USB stick, then a computer that started up in two different systems, and every install and tweak was an engineering nightmare. I spent more time fixing the set-up than watching anything learn.
Then, being me, I got swept off by other sudden enthusiasms for a few years. My memory's not the best, and I've a habit of saving over old files, so the early chapters are a bit hazy.
In September 2026 I picked it back up, and went at it like a man possessed. I made the mistake of carrying on exactly where I'd left off, without looking at how far things had moved on, and being stuck in the past cost me ages. There's a folder on my computer called Do Not Use, and seven different versions of Track & Field. My Space Invaders bots worked out that the corner was the safest place on the screen, then sat there firing at nothing. A long list of failed experiments, then the breakthrough. I made most of it up as I went along.
It started out running in the Linux that's built into Windows, and then I thought: I'm going to make a portable version, so anyone can try it and get involved. No Linux. Nothing to install. Unzip it, double-click it, pick a game.
I'm sure there are people out there with more talent for computers and AI than me who've been put off by all the setting up. This is for them. If it gets one of them hooked, it's done its job.
It must run in the genes. My daughter's just left university after studying machine learning, and now she works in AI herself. Trying to explain my old evolving bots to her is half the reason this exists. The other half is my retro arcade club. We meet once a month, and I promised I'd have something to show them.
Cheers, Jonny
FOR THE CURIOUS: HOW CHIP ACTUALLY LEARNSNo equations, promise. Just how it works, the way I'd explain it over the counter.
He looks at the screen, like you do
Chip doesn't get any secret information from the game. He sees the same picture you'd see, shrunk down, plus the score. He looks at the last few pictures together, like a flick-book, so he can tell which way the ball's going and where the bullets are heading.
He has a go, then looks back
He plays for a little while, then pauses and has a think: did that go better or worse than I expected? Whatever went better, he does a bit more of. Whatever went worse, a bit less. Never too much at once, so he doesn't forget what he already knows. Then he plays again. Millions of times.
It's like a boxer watching the tape of their last fight. Same boxer next time, just a better one.
He learns to be patient
My early bots only cared about points, so they grabbed the quickest points they could. In Space Invaders that meant sitting in the corner and firing at nothing. The fix was making a lost life really count. Camping in the corner gets you shot more, so he learns the only way to keep his lives is to move, dodge and get good. Points now are nice. Still being alive later is better.
Get the balance wrong the other way and he turns coward. One night I paid him a little for every second he stayed alive and went to bed. By morning he was hiding under the bases, not firing at all.
He gets curious
In some games the points are miles away. A bot that only chases points never finds them, so it just wanders about. Pick the exploration setting and Chip gets a little reward every time he finds a screen he's never seen before. Curiosity, basically. It's his best chance in something like Pitfall.
Before the breakthrough: breeding bots
I bred them first. An Atari has 128 bytes of memory, so I made each bot a list of 12,800 numbers (12,806 if you're counting). I liked the symmetry. Each bot picked one of six moves at a time. All 896 of them played three games each, the best survived, and the next lot were copies of the winners with every number nudged slightly at random. They learned fast, then hit a wall. Well, a corner.
Evolution mode: a family of Chips
The kit's second way to learn is the grown-up version of my old breeding idea. A small family of Chips take turns practising. After each round, the weakest gets swapped for a copy of the best, with a few of its learning settings nudged up or down at random. I call those nudges mutations. No idea if that's the right word. The hope is a monster in a surprisingly short time; I haven't raced it against the one-brain version yet.
He can learn by watching
Both ways of learning can learn by watching you play. Show him how, and next time he trains he copies you first, then learns to beat you. You can even let one bot show a fresh Chip how it plays. When I tried that, he seemed to fly ahead, and the teacher's style stuck around long after. Part of that's by design (the pull towards the teacher fades slowly, over 5 million moves), so I want to measure it properly. It still baffles me.
What's next: playing in his head
The cleverest AIs now work out how a game works, then practise inside their own imagination, like a snooker player lining up shots in their head the night before a match. Then there's learning from videos of experts playing. That's where I'm heading next. No promises on dates.
THANK YOU: THE CLEVER PEOPLE AND PAPERS BEHIND ITI'm self-taught and I learn as I go. These are the people whose work I've read, borrowed from or bumped into along the way.
- DeepMind · 2015Human-level control through deep reinforcement learning
The one that started it all for me. A computer learning dozens of Atari games from nothing but the screen and the score.
- OpenAI · 2017Proximal Policy Optimization
Chip's way of practising: have a go, look back, do a bit more of what worked, but never change too much at once.
- DeepMind · 2018IMPALA
Where Chip's eyes come from: the part of his brain that looks at the screen.
- Uber AI · 2017Deep Neuroevolution
Proof that breeding the best bots can hold its own on Atari games. Read loads on it.
- Uber AI · 2019Go-Explore
Clever ideas for games where the points are miles away and you have to go exploring first.
- OpenAI · 2018Exploration by Random Network Distillation
Where Chip's curiosity comes from: a little reward for finding screens he's never seen.
- NVIDIA · 2019CuLE: Atari games on a graphics card
Running thousands of Atari games at once on a graphics card. Blew my mind.
- DeepMind · 2018Playing hard exploration games by watching YouTube
An AI that learns from videos of people playing. Next on my list.
- DeepMind · 2019MuZero
Works out the rules for itself, then plans its moves in its head.
- Danijar Hafner and friends · 2023DreamerV3
Dreams up the game in its head and practises in its dreams. One day, Chip.
- The Farama Foundation and friendsArcade Learning Environment
The Atari 2600 playground AI researchers use. The kit uses it for the Atari games.
- The emulator teamsFinalBurn Neo and MAME 2003-Plus
Years of volunteers' work keeping arcade machines alive. They run the arcade games inside the kit, under their own licences.
- Everyone behindPyTorch and Python
Free tools that do the heavy lifting under the bonnet.
And thanks to my retro arcade club, for listening to me go on about it.
