You will be redirected back to your article in seconds

The Neuroscience of Mind-Control Gaming

In late 2016 Boston-based startup Neurable, the brainchild of neuroscientist Ramses Alcaide and a collective of fellow alumni from the University of Michigan, received a $2 million investment to develop software that will make it possible for users to control virtual and augmented reality with their mind. In other words, a “brain mouse” connecting your intentions to a computational device. Think right, go right. Think left, go left.

Neurable’s first project, the short video game demo “Awakening,” is a proof of concept for this thought experiment. “Awakening” is played using a VR headset fitted with an internal cap of electrodes that comb through your hair and down to the scalp, reading the electrical activity that is occurring at the level of your cerebral cortex – the top layer of your brain. In regular bursts, patterns of electrical activity begin to emerge. Neurons firing in wave-like unison are detectable through Electroencephalogy (EEG), a non-invasive method of measuring their voltage fluctuations.

“What we do is look for discrete brainwaves that we then leverage for control,” says Neurable’s Adam Molnar.

Imagine three items in front of you. As you focus your attention on a toy block it begins to pulse with light. As it does, your brain subconsciously registers its particular pattern of flashes. And then certain neurons begin to “fire” in response. Neurable’s software processes this noisy EEG data then finds the signal within it and translates it into a game command: That’s the one I want.

It isn’t science fiction, although it’s fitting that you play as a character who wouldn’t be out of place in one.

“Awakening” starts with a warning over the speaker system: “Wake up. This is not a test.” You play as a psychokinetically-gifted child and government prisoner – think Eleven from “Stranger Things.” You’re locked inside an unassuming examination room that contains a few toys scattered nearby: A block, a balloon dog, and a ball. These light up in a pulsating beam, each rising into the air, slowly rotating until you turn your focus to the next. Hanging from the opposite wall is a mirror which, if smashed, will reveal a hidden keypad and a way out. The aim is to escape the room. The trick? No hands, only thoughts.

Neurable’s mind-reading technology is better thought of as a simple brain-reading machine. Its VR headset appears just a little more alien than your standard-issue HTC Vive, outfitted with six bulky electrodes which detect the wearer’s basic intentions. The signal that emerges from such a brain-computer interface – a BCI for short – can’t tell if you’re hungry or if you’re visualizing your bank PIN, but it can capture the user’s intent from a constrained set of options.

“The interaction method works like a binary click,” says Molnar. “So either ‘yes’ or neutral, very much like a computer mouse.” But an EEG isn’t necessarily the most robust technology to work with.

“There are many difficulties working with EEG,” Molnar continues. “It is difficult to make sense of brain-activity that propagates from the neurons speaking to each other, through the skull, through one’s scalp, and just barely into the EEG sensor. Generally, EEG data is very noisy in the sense that it is very hard to get a clear signal for what you are looking. Anything from a smile to an eye-blink (called muscle artifacts) can mess with the EEG signals that are already difficult to pick up.”

It’s for this reason that most laboratories use “wet electrodes,” or electrodes dipped in a conductive gel, to enhance conductivity and pick up clearer EEG signals. However, Neurable has been experimenting with various methods of improving the process while using dry electrodes, which unlike their gel counterparts can be used more easily outside of a lab setting.

“Neurable has developed proprietary algorithms that deal with signals processing, which allow us to understand brain-activity, with less data, more reliably,” Molnar continues. “Because of this, we can use fewer sensors that are dry, so without the laborious task of applying conductive gel, while allowing low to moderate levels of physical activity, i.e. walking, moving your head, something that is quite difficult for most other EEG applications.”

When the company first began, they were using a cap with 32 wet electrodes to do the job, in a process that required 30 minutes of calibration – a system of rehearsing mental exercises so that the AI program can successfully classify your brain patterns and generate the desired movements. More recently, Neurable has reduced this to six dry EEG sensors which requires a calibration time of just two minutes.

Neurable isn’t the only team testing the waters of BCI. In the past few years, a number of tech industry stalwarts have turned their attention toward connecting minds and machines. In 2017, Facebook announced a team of 60 engineers are currently working on keyboard-less typing, through a device which scans your brain to detect your internal dialogue and translate it to text. Likewise, it was just in September that Elon Musk, under a ganja fog, told podcast host Joe Rogan of his plans to merge AI with the brain.

“How much smarter are you with a phone or computer or without? You’re vastly smarter, actually,” Musk said at the time. “You can answer any question pretty much instantly. You can remember flawlessly. Your phone can remember videos [and] pictures perfectly. Your phone is already an extension of you. You’re already a cyborg. Most people don’t realize you’re already a cyborg. It’s just that the data rate … it’s slow, very slow. It’s like a tiny straw of information flow between your biological self and your digital self. We need to make that tiny straw like a giant river, a huge, high-bandwidth interface.”

But in spite of all this hopeful futurism, BCI is likely still a few decades out from bringing our cyborg fantasies to life. “BCI needs to be integrated into your body for you to become a cyborg,” says Dr. Heather Read, a neuroscientist and professor in Psychological Sciences and Biomedical Engineering at the University of Connecticut. By comparison, EEG-based BCI is more similar to riding a bicycle. It is an extension of ourselves hurling us faster than we could run. “No one thinks twice about using a bicycle as an extension of your own body to move forward faster.”

It may be surprising that the technology necessary for bridging the wall between mind and machine have existed in some form since an era of rotary dial telephones and computers the size of living rooms. The term brain-computer interface was first coined in 1973 by Jacques Vidal, Emeritus Professor and founding member of UCLA’s Computer Science department, who posited a direct communication pathway could be created between the brain and an external device.

“EEG has been around for decades,” says Stanford neuroscientist Sharlene Nicole Flesher. “I honestly don’t expect to see any improvement to things like EEG tech in the next 10 or even 20 years. While the advent of machine learning and AI can sort of help, the signals available from EEG are probably at or near their peak performance right now.”

Since 2017, Dr. Read worked as director of the school’s Brain Computer Interface Core, where in collaboration with MIT, her team is developing a technology that can read-out and modulate human brain activity with timing precision 100 times faster than the human reaction time. Unlike Dr. Flesher, she maintains the ability to read out brain and behavioral activity will get faster and more accurate over the next two decades.

“This timing precision will make possible a whole new area of real-time neuromodulation to augment learning while using VR/AR and could be used to provide neuromodulation therapy,” she says. “Combined with high-speed wireless interfaces this will help make BCI vastly more fluid, effective and fun.”

But for both Flesher and Read, the true future of brain-computer interfaces is within-head, with further experimentations into brain-computer interfaces which can be implanted into the brain.

“The technical improvements I expect to see [in the future] are in electrode design which would make the idea of having something in your brain less crazy,” says Flesher. Indeed, the idea of putting something into your brain for fun does seem like a big ask. Invasive brain surgery remains the domain of the medical community in 2018 and modern BCI implants are designed to help restore lost critical functions. But according to Flesher, invasive brain implants are likely to be adopted by the entertainment industry. “The idea of implanted sensors for things like gaming/convenience is maybe on the scale of 40-50 years away,” she says.

According to Read, we might not even have to wait that long for our telekinetic cyborg powers.

“With enough vision and funding in 20-30 years, when the benefits outweigh the risks, it will be feasible to have sensors surgically implanted inside the brain and body allowing you to ‘telekinetically’ control devices and cars and other machines.”

More Gaming

  • Classic Cel-Shaded Shooter 'XIII' Being Revived

    Classic Cel-Shaded Shooter 'XIII' Being Revived With 2019 Remake

    Microids and PlayMagic LTD are bringing a remake of the cult classic cel-shaded shooter “XIII” to consoles and PC this November. The game’s first teaser trailer debuted on Thursday (April 18), showing off a small tidbit of what players can expect from the revitalized series. The game will follow “Thirteen,” a soldier who’s lost his [...]

  • 'EVE Online' Developer Investigating Ban of

    'EVE Online' Developer Investigating Ban of Real-World Politician

    “EVE Online” developer CCP Games is investigating its own ban of a real-world politician from the massively multiplayer online role-playing game, it revealed in a statement on its website Wednesday. “Last week, on Apr. 8, 2019, we announced the permanent banning of Brisc Rubal and their ejection from the Council of Stellar Management,” CCP Games [...]

  • 'Cuphead': The Imperfections of a Perfect

    'Cuphead': The Imperfections of a Perfect Port

    “Cuphead” arrives on Nintendo Switch this week. By all accounts, it appears to be a near-perfect port. And that’s what worries me. Though absolute parity across every gaming platform does not technically exist, it sure feels like it does. And though much is gained by the relative ease of producing exact replicas of hit games [...]

  • Fortnite Battle Royale

    'Fortnite' Dominates Twitch Again as 'Apex Legends' Viewing Wanes

    “Fortnite” is back on top in Twitch viewership, yet Ninja didn’t hold the title of most watched streamer in the past few months, according to a report released Thursday from StreamElements. The State of the Stream report from StreamElements detailed the first quarter of 2019 in Twitch and YouTube video game streaming stats. “Fortnite” saw [...]

  • Fortnite Battle Royale

    'Fortnite' Season Eight, Week Eight Challenges Guide

    Season eight’s week-six challenges are live in “Fortnite,” and that means new ways to earn Battle Stars and experience. By collecting more Battle Stars and upping their tier, up to a season maximum of 100, players can earn limited-time rewards like cosmetic outfits and XP boosts. (Collecting 10 Battle Stars will unlock one additional tier.) [...]

  • 'The Walking Dead: The Telltale Definitive

    'The Walking Dead: The Telltale Definitive Series' Is Available for Pre-Order

    “The Walking Dead: The Telltale Definitive Series,” a comprehensive bundle featuring all four seasons of the narratively-driven zombie game, is now available to pre-order for Skybound Insiders on PlayStation 4, Xbox One, and PC, developer Skybound Games announced Thursday. Pre-orders will open for the rest of the public starting Friday and running until May 31. [...]

  • Konami Confirms Full 'Castlevania Anniversary Collection'

    'Castlevania Anniversary Collection' Lineup Includes These Eight Titles

    Konami revealed the final four games in the first “Castlevania Anniversary Collection,” which contains eight titles pulled from the popular series. The latest titles to be revealed include “Castlevania: The Adventure,” “Castlevania II: Simon’s Quest,” “Castlevania Bloodlines,” and “Kid Dracula.” The full collection contains eight titles, including “Castlevania,” “Castlevania II: Belmont’s Revenge,” “Castlevania III: Dracula’s [...]

More From Our Brands

Access exclusive content