“I want to solve AI,” said Joseph. The problem of artificial intelligence is a hard one. A lot of people have worked on it, to varying degrees of success. Why is it hard? “Take, say, you. You see things using your eyes, and you make a mental model of the world. What would that mental model look like? No one has got a really good mental model that can do all of the things the human brain can do.”

There are some basic recognition algorithms developed just recently that can, for example, recognise familiar objects. “But ask people in neural networks to tell you how their brains will describe justice, or truth. We don't have good models of these kinds of things. We don't just need a good model of that, we need an algorithm which can learn that. Which is hard.”

Joseph wants to solve that problem, but it's not something that can be solved straight up. The approach Joseph is taking is to solve a similar problem that has a lot of the same properties: prediction.

So what do we mean by prediction? “You've got an agent, some robot, which is receiving input from the world over time, say from a camera. What it has to do is work out what its input is going to be in the future.” To do that, it has to work out how the world it's observing works. The agent will be rewarded based on how well it makes predictions. It needs to be able to learn the way the world works.

There are a lot of existing approaches to solving this problem, but “a lot of them make stupid assumptions,” said Joseph.

“For example, they assume that the world is a finite state machine, but that there's no similarity between different, very similar states.”

To have intelligent agents interacting with a world, first they have to understand the world. “How can you tell how well you understand the world? Because you can make predictions about how it's going to act in the future.” Once you can make predictions about how the world will act, you can add in imagination: “if I did this, what would happen? If I did that, what would happen?”

“Ask people in neural networks to tell you how their brains will describe justice, or truth. We don't have a good model.”

The agent can use its predictions about the world to be able to tell what would happen in different situations. It can use its results to make optimal choices, because it can predict which choice will be the best. “Once you've got an image in your head of how the world works (which is what we do as humans), you can start to make good choices.”

The most popular method existing of trying to predict the way the world works is to treat it as a state machine: the world is in some state, and from that state it

will transition to some other state. “Treating the worldas a state machine has some problems,” said Joseph. “I take a room and I subdivide it into a 5 by 5 grid, so there are 25 squares. Then I close my eyes and feel around me, and my observation is whether or not I can feel walls. Your system is given which direction you went (North, South, East, or West) and an observation (which walls you can touch at any time). You don't understand motion—all you know is that you're in a state machine, and you're making transitions from grid square to grid square and feeling walls.”

“Learning that state machine is hugely, incredibly much harder than learning how motion works, and then that you're in a room which is 5 by 5.”

To learn the state machine that represents a 10 by 10 room—100 states—even the best algorithms can take more than 200,000 steps. Obviously, a person would be able to learn that faster. “How would we learn that faster? We recognise patterns. We don't assume that every state is completely unique: we assume that there will be similarities.”