Xbox’s Matt Booty Asked Microsoft To Invent AI Bot QA Testers For Games – Forbes

Sometimes you read a quote and you can just see the firestorm it’s about to set off, and that’s happened today with Head of Xbox Game Studios, Matt Booty, who “dreams” of using AI to QA test video games in the future, so much so he’s actually asked Microsoft to help him figure out how to do that.

Booty, speaking at PAX West, had a whole lot of thoughts on the idea of AI-based QA testing, given the demands of modern games, but his comments are bound to generate controversy given that QA has long been an underpaid, underappreciated part of game development, and the mistreatment has caused unionization to spring up at certain studios as a result as of late, including ones Microsoft is about to own.

Here’s what Booty says about the QA problem and the potential AI fix, via VGC:

“Some of the processes we have, have not really kept up with how quickly we can make content. One of those is testing.”

You think about a game, one of the biggest differences between a game and something like a movie, is if we’re working on a movie and you come in and say ‘hey, this ending, let’s tighten this up, let’s edit this, let’s cut that scene’, it usually doesn’t break anything at the beginning of the movie.

“But in a game you can be ready to ship, and a designer’s like, ‘I’ve got this one little feature, I’m just going to change the color on this one thing’’ and then it somehow blows up something and now the first 10 minutes of the game doesn’t play. So that testing aspect, every single time anything new goes into a big game the whole game has to be tested, front-to-back, side-to-side.

“My dream – there’s a lot going on with AI and machine learning right now, and people using AI to generate all these images. What I always say when I bump into the AI folks, is: ‘Help me figure out how to use an AI bot to go test a game.’

“Because I would love to be able to start up 10,000 instances of a game in the cloud, so there’s 10,000 copies of the game running, deploy an AI bot to spend all night testing that game, then in the morning we get a report. Because that would be transformational.”

Booty is right in that testing is incredible tough in games, and what we consistently see is a lot of big titles launching full of bugs, not necessarily because anyone did their job wrong, but because millions of players will do more “testing” time of the final version of the game in the first few hours and days than the entire QA team could do in months or even years. This has resulted in some uncomfortable situations, like CDPR seemingly blaming testers for not catching Cyberpunk 2077 bugs, pre-launch, and debates about how true that was. But the scale problem is real.

But is AI the solution? That’s bound to be even more controversial. The human side of the argument would be getting studios to shell out for more QA testers, pay them better, curb their hours so they are more functional at work and better able to catch bugs. But here, it sounds like Booty is suggesting the automation of the process, and if you do invest a quality AI QA testing system, that seems like it would have the potential to put an entire chunk of the industry out of work. Not that anything like this actually exists in a meaningful way yet. But I know many will take Booty’s “my dream” comments here to be somewhat problematic given the plight of current QA workers. Also referencing AI image generation, where programs like Midjourney are currently upsetting a lot of human artists, may not be the best point of comparison here.

We’ll see if Booty clarifies his remarks, which he seems likely to, given the conversation this has generated, but his initial statement is also pretty emphatic about where he’d like to see the industry move, if possible.

Follow me on Twitter, YouTube, Facebook and Instagram. Subscribe to my free weekly content round-up newsletter, God Rolls.

Pick up my sci-fi novels the Herokiller series and The Earthborn Trilogy.