How I built my own Twitch-Plays-Pokémon: Part 2
I gave this talk at Baruco 2014.
I'm publishing it now as a series of blogposts. Make sure to check out the first part.
You can find the slides in my speakerdeck. You can also see the recording in YouTube.
How can I make my own: Problems of the 1st approach
Ok, so getting my MVP was easy. I'm almost ready to get funded!
But before talking to VCs there are some problems that I need to solve:
Simulating keystrokes is a rather weak way of controlling an emulator.
If by some reason the focus shifts to another window, the key-presses will not be sent to the emulator, which would make the whole system fail.
The system requires a graphical environment to run.
Because the emulator is meant for people to use it in their desktop environments, it outputs the GameBoy screen to the computer's screen.
That means that we actually need a graphical environment, something that's not
common in servers.
It also means that we have to position the emulator in a
specific position of the screen and then record that specific portion.
Requiring a desktop environment just makes everything much more complicated.
Recording the screen adds a huge latency.
If you send a command through the chat, because of video buffering, there's usually a 20 second delay.
That means that the button that you're pressing was for a screen that happened 20 seconds ago, which is far away from being acceptable. The original TPP has this same problem and this is one of the reasons that motivated me to make my own.
All this problems have something in common: they're caused by the emulator.
The ideal emulator for us would be one that:
- is headless, so it doesn't require a desktop environment.
- streams the output online
- has an API that allows to send the button presses directly, instead of simulating a keyboard.
I couldn't find any that does that, so I went back to my cookbook.
There was a small notice at the bottom that said:
"If you need a headless emulator, go the next page."
I flipped the page and there it was!
Ruby Version: What's an emulator
Before telling the story on how to cook the headless emulator, let me first explain what an emulator does.
A GameBoy emulator is a program that allows us to run GameBoy games in a regular computer.
The cool thing about them is that they run the exact same binary that's in the original GameBoy cartridge.
The only way of running the same binary, at least that I know,
is achieved by faking the GameBoy hardware.
That means, writing a CPU, a GPU, a memory controller and pretty much everything
inside the console.
When you start a GameBoy, the game is loaded into the RAM. When you start an emulator, the GameBoy ROM is loaded in a big array, which emulates the RAM.
The CPU loads the first byte of the RAM and looks it up in a so-called Instruction Table.
The Instruction Table is "hardcoded" in the CPU and it's a list of very tiny
instructions which are the ones that the CPU is capable of doing.
For example, the instruction equivalent to the byte 0xC0 is going to jump to another part of the program.
In our emulator, we have the same instruction table.
The core of the CPU emulation happens in what I call "the" Loop.
The CPU has an internal number called program counter, which tells it what's
the next byte to read.
The CPU reads the byte, then looks up in the instruction table what's
the corresponding operation for that byte.
Then, it executes the instruction for that byte.
And finally, it increases the program counter by one.
class Emulator
def step
next_byte = @ram[@program_counter] # => 0xC0
next_instruction = OPS[next_byte] # => :jp_nn
@cpu.send(next_instruction)
@program_counter += 1
end
end
This happens hundreds of thousands of times per second.
This loop is the basic principle on how computers work, and before working on this project I had no idea about this.
To me, what's cool about all of this is that when people study computer science,
they learn this by the books.
They start with bits and bytes, then how CPUs work, and later on C.
This is very progressive. You start from a very low level programming language
and then go higher into Ruby.
In my case, it was just the opposite.
I started with high level computer languages and went down.
That's why names may sound weird if you have an academical background,
because those names are what those things mean to me.