CICERO: The first AI to play Diplomacy at a human level | Meta AI

There’s always been this promise with GameAi’s that one day they’re going to go beyond the Game Board into the real world. Games have served as benchmarks and challenge problems for AI going back decades. In 1997 you have Deep Blue famously beating Gary Kasparov in chess. In 2016 you famously have AlphaGo beating Lisa Dahl and Go. In 2019 we managed to make an AI that could be top humans in poker as well. All that was impressive but we want AI to go beyond just moving pieces on a board. At Meta AI we built Cicero, the first AI to achieve human level performance in the strategy game diplomacy. Diplomacy is one of the most intricate strategy games ever created. It’s a game about people rather than pieces. The complexity of the game comes from negotiating with other players and trying to work with them to achieve shared goals. Each turn in a game of diplomacy takes about 15 minutes where you negotiate with the other players, write down what you want to do, and then everyone reveals what they’re doing all at the same time. Diplomacy presents an enormous challenge for natural language generation. In order to play the game the AI agent has to converse simultaneously with six other players sending hundreds of messages over the course of a game. The agent has to use natural language to form relationships and build alliances with other players. What impresses me most about Cicero is its ability to communicate with empathy and build rapport whilst also tying that back to its strategic objectives. Its strategy informs its communication and its communication informs its strategy. You can think of Cicero as having two components, a planning engine and a dialogue component. And these two components work together to produce meaningful dialogue. The planning engine will generate a plan for itself and for its partner that it thinks are strong ideas. And then it will feed that into the dialogue agents that will then send a message corresponding to those plans. Cicero followed quite a similar learning path to a lot of diplomacy players. At first just because the game lets you tell a lie, you tend to do that. But one of the great learnings and one of the great things about diplomacy is it teaches you that being honest is the best way to succeed. Today’s AI assistants are good at simple quest or answering tasks like telling you the weather. But imagine a future where they could maintain long-term conversations with the goal of teaching you a new skill. There’s a lot of practical applications for any AI agent that is fundamentally honest and fundamentally collaborative. Imagine a video game in which the non-player characters could plan and converse like people do, understanding your motivations and adapting the conversation accordingly to help you on your quest. Cicero is not about the game of diplomacy. We’re using diplomacy as a benchmark to measure our progress. We’re open sourcing the code and models for Cicero so that others in the community can build on top of this research responsibly. It’s really cool that we manage to make AI’s that can be humans at games, but ultimately we want AI’s that can cooperate with humans, understand them, and work with them. There’s lots of exciting challenges still to come for Cicero.