AI Ethics, Computer With Souls, Self-Playing Games

From Galatea to GLaDOS, our cultural fascination with not-quite-human consciousness and intelligence has spanned thousands of years. At PAX East on Saturday, Dr. Tyr Fothergill and Dr. Catherine Flick reflected on the place of AI in video games – both how it’s depicted in series like “Mass Effect” and how the technology is increasingly used within the industry in reality.

According to Flick, a scholar of computing and social responsibility at De Montfort University, the stories we tell about artificial intelligence in games reflect our values and attitudes toward AI as a whole. As she explained, “AI is usually viewed as operating within the context of human morality.” Eric Walpole, who wrote the character of GLaDOS from “Portal,” had one essential rule for her dialogue: She shouldn’t talk like a computer. While Flick describes GLaDOS as “notoriously evil,” those glimpses of something like humanity complicate our understanding of her role. She might, Flick suggests, simply be following her programming, prioritizing the perfection of the portal gun over her test subject.

Frank Lantz’s viral clicker game “Universal Paperclips” takes the same anxieties to a new extreme when an AI tasked with maximizing paperclip production determines that the most efficient way is to eliminate life on Earth. As Flick put it, “This is a lesson about the dangers of creating super-intelligent machines without programming protections for human existence into them, and the dangers presented by a lack of human values.”

In the “Mass Effect” franchise, the arc of the mechanoid species known as the Geth can also be taken as a cautionary tale. After recruiting Legion, a Geth unit who’s evolved beyond the constraints of his original programming and achieved self-awareness, the player must decide whether the rest of his race should be liberated at the cost of their “organic” oppressors.

For Fothergill, a research fellow with the Human Brain Project who specializes in human-nonhuman relationships, fictional beings like the Geth beg the question: “What is it to be human? Is it consciousness or, as Legion said, a soul?” According to one philosophy, known as “strong AI,” it’s simply a matter of information processing. If a computer can be programmed with the same inputs and outputs as a human brain, the argument goes, that computer has a mind in the same way we do.

This is a highly controversial take on consciousness – but regardless of how we define the mind, it’s clear that real-world AI is progressing rapidly. Google’s AlphaStar AI made headlines earlier this year when it defeated top-tier professional gamers in “StarCraft II,” a complex strategy game that demands both instantaneous decision-making and long-term tactics to succeed. Even when handicapped so that its reaction times were slower than its human competitors, AlphaStar dominated 10-1. Its accomplishments are a significant step up from those of IBM’s Deep Blue, which made history when it defeated reigning chess world champion Garry Kasparov in 1997. But some of the same questions still apply: the AI may be good at beating the game, but is it really playing it?

“AlphaStar doesn’t derive any fun, so far as we know,” said Flick. Nor would it be fun for the average human player to compete against – “especially the first ten times it kicks your arse.” Like other AI now learning to “play,” its understanding of the task it’s been assigned can lead to some baffling decisions by a human player’s standards. Fothergill described one case in which an AI was programmed to win a boat racing simulation – and did so by ignoring the race entirely, instead driving around in circles to rack up bonus points for stunts.

Sometimes, these imperfections are deliberate. When it comes to enemy NPCs, Flick noted, “We want fights to be challenging, but not frustratingly so. If [mob AI] were perfect – if it shot you in the head every time you stuck your head above the parapet – that wouldn’t be much fun.” And in the case of in-game allies, “We don’t want them to steal our glory by rushing up to the boss and killing them and taking all of the credit, so we probably have to dumb them down in terms of their accuracy and productivity.”

The goals and datasets these engines rely on are essential to the outcome – AlphaStar now has the equivalent of hundreds of years of experience with “Starcraft II,” but it was initially trained by “supervised learning” from real human matches. Increasingly, the game industry itself is learning from players, too. “Companies create profiles of your style of gameplay,” Fothergill explained. “The cutting edge of gaming AI is pointing toward a customized gameplay experience, particularly within role-playing games.” Data including the choices you make in an RPG, when you play, and even when you quit can be used for “more than you might expect… and may potentially be used against you if it ends up in the wrong hands.” As the panelists noted, player data from war games is already being used by the military to identify potential recruits.

Where artificial intelligence goes from here remains an open question. Fotherhill joked that we often model our AI on ourselves “because we’re really bad programmers” – and noted that being trained on human behavior means inheriting human biases, too.

Despite our fixation with simulating personhood, there are also aspects of the human experience we may never be able to convey. “How do you program a goal like ‘have dreams’ or ‘have a life’?” Fotherhill asked. In games and in reality, our creations are only as human – and humane – as the data we give them.

Popular on Variety

More Gaming

  • Telling Lies - Logan Marshall-Green

    Sam Barlow's 'Telling Lies' Government-Surveillance Thriller Game Sets Release Date

    After more than two years in the works, “Telling Lies” — the investigative thriller from acclaimed game creator Sam Barlow — is ready to take the stage. The game, produced with and released by Annapurna Interactive, will be available next Friday, Aug. 23, via Steam and Apple’s Mac and iOS app stores. “Telling Lies” will [...]

  • David Messinger, Activision Blizzard

    Activision Blizzard Hires CAA Veteran David Messinger as CMO

    Activision Blizzard tapped David Messinger, a former 15-plus-year veteran of CAA, as its first corporate-wide chief marketing officer. Messinger, based in Santa Monica, reports to Coddy Johnson, Activision Blizzard’s president and COO. It’s the first time the video-game company has appointed a CMO who will oversee the global marketing operations across all of Activision Blizzard [...]

  • Editorial use onlyMandatory Credit: Photo by

    Oculus Co-Founder Nate Mitchell Is Leaving Facebook

    Oculus co-founder Nate Mitchell is leaving Facebook 7 years after launching the first Oculus Rift headset with a Kickstarter crowdfunding campaign. Mitchell announced the move on Reddit Tuesday, calling it “bittersweet news.” Mitchell said that he would take some time off to spend with his family. And while he committed to still being involved in [...]

  • nomadic seoul

    Nomadic Expands Abroad, Opens VR Center in Seoul

    Location-based virtual reality (VR) startup Nomadic is getting ready for an international expansion: The company will open its first VR center outside of the States in South Korea later this month. The new location, housed in the CGV Gangbyeon multiplex in Seoul, is being opened in partnership with theater chain CGV and technology provider CJ [...]

  • Singularity 6 - Anthony Leung and

    ‘Meaningful’ Games Start-Up Singularity 6 Raises $16.5 Million Round Led by Andreessen Horowitz

    Singularity 6, the start-up founded by ex-Riot Games developers that’s aiming to create a community-oriented game where players “can feel like they are valued and belong,” closed $16.5 million in Series A financing. The funding round was led by venture-capital firm Andreessen Horowitz, with participation from seed investor London Venture Partners (LVP), which previously put [...]

  • Activision Q2 2019 Results Above Expectations,

    Activision Blizzard Q2 Beats Outlook, But Not Last Year's Second Quarter

    Down may not be the new up after all: Activision Blizzard celebrated its Q2 2019 earnings as a success story Thursday, with CEO Bobby Kotick saying: “Our second quarter results exceeded our prior outlook for both revenue and earnings per share.” But while the results were considerably better than the company’s previous outlook for the [...]

More From Our Brands

Access exclusive content