AI researchers have built a Minecraft bot that can explore and expand its capabilities in the game’s open world — but unlike other bots, this one basically wrote its own code through trial and error and lots of GPT-4 queries.

Called Voyager, this experimental system is an example of an “embodied agent,” an AI that can move and act freely and purposefully in a simulated or real environment. Personal assistant type AIs and chatbots don’t have to actually do stuff, let alone navigate a complex world to get that stuff done. But that’s exactly what a household robot might be expected to do in the future, so there’s lots of research into how they might do that.

Minecraft is a good place to test such things because it’s a very (very) approximate representation of the real world, with simple and straightforward rules and physics, but it’s also complex and open enough that there’s lots to accomplish or try. Purpose-built simulators are great, too, but they have their own limitations.

MineDojo is a simulation framework built around Minecraft, since you can’t just plonk a random AI in there and expect it to understand what all these blocks and pigs are doing. Its creators (lots of overlap with the Voyager team) put together YouTube videos about the game, transcripts, wiki articles, and a whole lot of Reddit posts from r/minecraft, among other data, so users can create or fine-tune an AI model on them. It also lets those models be evaluated more or less objectively by seeing how well they do things like build a fence around a llama or find and mine a diamond.

Voyager excels at these tasks, performing much better than the only other model that comes close, Auto-GPT. But they have a similar approach: using GPT-4 to write their own code as they go.

Normally you’d just train a model on all that good Minecraft data and hope it would figure out how to fight skeletons when the sun goes down. Voyager, however, starts out relatively naive, and as it encounters things in the game, it has a little internal conversation with GPT-4 about what it ought to do and how.