×
You will be redirected back to your article in seconds

Valve Psychologists Have Big Questions, Few Answers About Brain-Controlled Games

When it comes to future of how our brains will control (or be controlled by) video games, there’s only one thing Valve knows for certain: it wants to measure as many minds as possible.

Mike Ambinder, Valve’s Principal Experimental Psychologist, gave a talk at the Game Developers Conference on Friday that outlined what he believes are the many possible futures of the way that we play in terms of Brain-Computer Interface (BCI). Currently, we mere humans playing video games have an intent, then use physical actions (like button presses or head moving) to interact with the game and the game responds. BCI explores what would be possible if the human interaction was eliminated from this clumsy pipeline, essentially turning intent into gameplay or vice versa.

“How will you interact with technology in 10 years?” he asked, as he laid out many possibilities for how BCI could be implemented, measured and used by game designers to create enhanced capabilities in their titles. “If you think about having a conversation, there are verbal and nonverbal parts. When you get those parts together you get a complete conversation. We might be missing the nonverbal parts of the conversation.”

Beyond the most common interpretation of using BCI to control what’s happening in a game, he talked about seeing a future where BCI could continually measure players, where games could adapt to the players’ cognitive states and even where games could affect those states directly. He also referred to “The Matrix” a number of times.

Ambinder’s talk was almost entirely hypothetical. Though he detailed some of the tools that Valve is using to measure these brain and thought patterns, he gave very few actual innovations or information about gaming interface changes that could come in the near or far future. Instead, he asked question after question about what could be possibly be achieved through speculative data, working towards theoretical benefits.

The most concrete implication for present and future BCI use, which he said Valve is currently using, is in play testing.

“What happens when you get additional information from the player that we’re not currently getting from data input?” Ambinder asked. “We would get an improved player experience If we had access to internal states and emotions and cognizents.”

Again and again, he called for the collection of vast measurements of player data, so developers can assess what a player is feeling when they play a game. He stressed the need for using that data to determine how to design a game to maximize the emotions that a developer wished to inspire.

“We simply want to understand locations of increased levels in the brain,” he said. “If we could take these patterns of activity and translate them to experiences, that’s the key.”

In Valve, Ambinder said they were using a variety of different tools and machines to take the measurements of players to begin collecting this data. Those tools include an electroencephalogram (EEG) headset and devices that measure heart rate, skin response, eye tracking, facial expressions and more.

Though the data, he said, “is noisy.” Which is why he believes that it should be massively collected, so researchers, developers, and assumedly others, can find trends in what the brain is experiencing.

“If we could get the advantages of direct observation and internal states, we’d be much better game designers,” Ambinder said. “We could always be playtesting on everyone.”

Valve, as a hardware development company, has been no stranger to experimenting with possible interface changes, or pushing forward the expectations of how players can interact with games. They also created the Steam controller, an unconventional input device for playing games that replaced controlling analog sticks with trackpads. Valve is also responsible for developing the Vive virtual reality headset, one of the leading technologies in the VR space.

As a possible way collect and track this data, Ambinder talked about putting sensors like the EEG in VR and AR headsets as a way to gather those measurements on a large scale.

“One advantage of VR and AR is you’re getting consistent contact with brain activity, so you might be able to do some interesting things,” he said.

One of the larger benefits of having these measurements, as he saw it, is through the possibilities of creating adaptive gameplay, where the game actually changes based on what the player was experiencing.

“Game dynamics would be no longer static if we had access to this data,” Ambinder said. “Just as players respond to the game, the games could begin to respond to the player. As a consequence, gameplay becomes adaptive and personalized.”

Essentially, he looked as collecting that brain pattern data and using it to deliver the type of game or game experiences that would attract players the most. He talked about measuring his own thought patterns while playing Valve’s “Team Fortress 2,” and seeing what happened to his brain waves when he collected a rare item or killed an opponent. The maximization of those sensations through gameplay is one of the possible eventualities of BCI as he sees it.

“Difficulty levels would become a thing of the past,” he said. “We’d have dynamic difficulty adjustment. Difficulty that’s designed for you.”

And then there is the method for crafting that interface between the brain and the computer. Ambinder said that the future of this technology could be non-invasive, like the EEG headset, or even invasive technologies like implants.

Which brings us to “The Matrix,” the completely simulated world featured in the late ‘90s sci-fi movie, which Ambinder discussed as being almost a foregone conclusion. He said all of this work, stimulating synaptic responses in order to elicit desired emotions or reactions, essentially ended in matrix-like experiences.

“It could be decades or centuries away,” he said. “But the path to that reality, people can see it and they’re actively working on it.”

However, Ambinder continuously maintained how hazy the future is about how BCI could evolve and whether all or any of what he considered in the talk would come to fruition.

“Neuroscience has a long way to go,” he said. “There’s so much we don’t understand. Like, how do you think a thought? How do you play a game.”

Of course, he would not give away Valve’s research secrets in a GDC talk. So, it’s entirely possible that the industry giant is crafting marketable tools to measure brain states, create adaptive games or to take us closer making “The Matrix” a reality. But for his presentation, Ambinder kept the whole topic extremely abstract.

“I’m optimistic, but we don’t know yet, we need to create something and see,” he said about some theoretical technology or implementation. “It can’t just be theory. We have to make something that actually works.”

More Gaming

  • harry potter wizards unite

    'Harry Potter: Wizards Unite' Off to Slower Start Than 'Pokemon Go'

    Niantic’s new location-based augmented reality game “Harry Potter: Wizards Unite” is off to a solid start — but it doesn’t seem to have quite the same momentum as “Pokemon Go” did when it launched 3 years ago. That’s according to initial estimates from app analytics specialist Sensor Tower. “Harry Potter: Wizards Unite” first became available [...]

  • survios at the linq

    Not Just Gambling: Las Vegas Is Becoming a Virtual Reality Hub

    Walk into the Linq on the Las Vegas strip these days, and you might not immediately realize you just stepped into a casino. Instead, you’ll stumble across a series of living-room-like lounge setups, complete with leather couches, big-screen TVs, Xbox Ones and Oculus Go VR headsets. There’s also a bar with a massive wrap-around touch [...]

  • Netflix HQ LA

    Netflix Engineers Developed a Rumble Pack Feature During Latest Hack Day

    A duo of Netflix employees had a unique idea for making the service’s shows even more stirring: They added a rumble pack option to the Netflix app as part of the company’s latest hack day. When watching shows like “Voltron,” the feature makes phones vibrate in sync with the action on the screen, similar to [...]

  • Niantic's Harry Potter: Wizards Unite Game

    ‘Harry Potter: Wizards Unite’ to Launch This Week

    “Pokemon Go” maker Niantic is getting ready to launch its next big game: “Harry Potter: Wizards Unite” is set to launch in June 21, the company revealed in a tweet this week. The worldwide launch of Harry Potter: Wizards Unite begins this Friday, June 21! Keep your eyes peeled and wand ready for more information [...]

  • Sarah Iooss - Twitch

    Twitch Hires Sarah Iooss, Former Mic and Viacom Exec, as Head of North America Sales

    Twitch has tapped Sarah Iooss, most recently EVP of revenue at millennial-news startup Mic, to lead the advertising sales team in North America for the Amazon-owned live-streaming video platform focused on video gaming. Iooss will be based in New York, reporting to chief revenue officer Walker Jacobs, who joined Twitch last fall after serving as [...]

  • littlstar ps4 app

    Littlstar Expands Beyond VR With New PS4 App, Signs Up 30,000 Paying Subscribers (EXCLUSIVE)

    A&E-backed immersive content aggregator Littlstar is expanding beyond virtual reality (VR): The New York-based startup launched a dedicated PlayStation4 app this week, giving users of Sony’s game console access to their personal media libraries, as well as both traditional and 360-degree video content from a variety of publishers. Littlstar also announced that it signed up [...]

  • Game of Thrones Season 8 Production

    'Game of Thrones,' Netflix VFX Among Those to Be Featured in SIGGRAPH Production Talks

    VFX pros behind the final season of “Game of Thrones,” the blockbuster film “Avengers: Endgame,” Pixar’s upcoming “Toy Story 4,” last year’s Oscar-winning “Spider-Man: Into the Spider-Verse,” Netflix series, including “Stranger Things,” and more will give SIGGRAPH 2019 attendees a behind-the-scenes look at their work during the conference’s Production Sessions. There will even be a [...]

More From Our Brands

Access exclusive content