×

Microsoft Has Been Quietly Working on the Future of Consumer AR

When it comes to flashy augmented reality (AR) glasses for consumers, all eyes are on two companies these days: Magic Leap, which released its Magic Leap One developer headset last summer, and Apple, which is reportedly getting ready to unveil an AR device next year.

One company that isn’t getting quite as much attention, despite having worked in AR for years, is Microsoft. That’s largely due to the fact that Microsoft’s Hololens headset has been positioned as an enterprise device, made for healthcare professionals instead of gamers. But make no mistake: Microsoft has had its eyes set on the consumer AR market for quite some time — and it is increasingly hinting at plans to show off its vision for how we all may use AR one day.

Case in point: When the Redmond-based software giant unveiled the second generation Hololens device last month, Microsoft’s Alex Kipman was joined by Epic Games CEO Tim Sweeney. The game industry executive was ostensibly at the event to announce the addition of Hololens support to Epic’s Unreal game engine to give developers another tool to create AR and VR content.

But there seemed to have been another reason why Microsoft had invited Sweeney on stage. “In the years to come, Epic will support Hololens in all of our endeavors,” Sweeney promised, all but confirming that Microsoft had planned a gamer-ready Hololens device for the near future. Microsoft executives have since repeatedly said that the journey to consumer AR will take years, not decades, further underlining the company’s commitment to the space.

How a consumer AR device made by Microsoft will look like, how much it will cost, and what it will be capable of are all questions that are hard to answer at this point. At best, we can look to the newly unveiled Hololens 2 for some clues.

That device, which Variety got to try at a recent briefing in San Francisco, is in many ways the logical evolution of the original Hololens. Like its predecessor, Hololens 2 is an all-in-one AR headset, capable of overlaying holograms over your view of the real world. It’s significantly more comfortable than the original — three times as comfortable, if we can believe Microsoft’s oddly specific math. And it features more than double the field of view of the original Hololens.

Microsoft’s Hololens 2 also comes with integrated eye-tracking technology, which introduces a range of new interaction models. In one demo shown off by Microsoft, an info card about another Hologram appeared in thin air. When your eyes reached the last line of the text, it scrolled up to reveal additional lines — a pretty magical experience.

But the biggest update may be advanced hand tracking, which makes it possible to touch and interact with Holograms in a whole new way. The original Hololens already featured some hand tracking, including tap and pinch gestures to select and rotate objects.

With its new hardware, Microsoft has significantly expanded the capability of such gestures, and is now tracking 25 joints in each hand. The resulting experience feels a lot more natural, allowing users to press buttons, grab Holograms, and even have AR hummingbirds land on the palm of their hands.

But Microsoft hasn’t just been working on making the Hololens more user-friendly. The company also used its event last month to unveil what it calls Spatial Anchors, its take on the AR cloud. As part of Microsoft’s Azure cloud computing infrastructure, Spatial Anchors allow developers to build persistent, shareable AR experiences that are tied to specific locations.

This could be used, for instance, to build multi-player AR games for mobile devices. And ultimately, the same technology will also able to power Microsoft’s own AR experiences, including those running on future versions of the Hololens.

The final leg of the stool for Microsoft’s AR efforts are the company’s Mixed Reality Capture Studios, which can be used to capture high-quality 3D holograms. Microsoft runs its own capture studio in San Francisco, and its technology powers two studios operated by partners in Los Angeles and London.

At the moment, Microsoft uses these studios to help publishers produce holograms for AR and VR experiences like Sir David Attenborough’s “Hold the World.” But for the company, the studios also function as a testing ground for a future commoditization of 3D capture technology. “This is teaching us how to get to a much more lightweight form factor,” said Mixed Reality Capture Studios general manager Steve Sullivan when he gave Variety a behind-the-scenes look at the San Francisco studio last year.

Right now, all of these efforts are just loosely connected. The Hololens, with its $3,500 price tag, is still clearly an enterprise device. Cloud Anchors are a developer tool, primarily geared towards mobile AR. And Microsoft’s Mixed Reality Capture studios are helping to produce content for AR and VR environments.

But taken together, all of these efforts hint at a future in which Microsoft wants to be able to do everything: Build mass-market AR devices with proven interaction models, power them with the necessary infrastructure for planet-scale AR experiences, and have enough content available to make them enticing for consumers. And when Apple and Magic Leap are going to vie for consumers to buy their AR hardware in the next few years, Microsoft may be ready to give them a run for their money.

Popular on Variety

More Gaming

  • Insomniac Games - Spider-Man

    Sony Is Acquiring Insomniac Games, Studio Behind 'Spider-Man' for PS4

    Sony Interactive Entertainment clinched a deal to acquire longtime PlayStation games developer Insomniac Games in its entirety, the company announced Monday. Financial terms of the deal were not disclosed “due to contractual commitments,” according to Sony. Insomniac Games is the developer of PlayStation 4’s top-selling “Marvel’s Spider-Man” and other PS games, including the “Ratchet & [...]

  • westworld awakening art

    HBO Launches 'Westworld Awakening' VR Game With Survios

    Ever wanted to enter the world of “Westworld?” Now you can — except it won’t exactly be like you might have imagined it. Instead of being one of those guests living out their fantasies, you will be a host, slowly coming to grips with your life being an elaborate illusion. “Westworld Awakening” has been produced [...]

  • Telling Lies - Logan Marshall-Green

    Sam Barlow's 'Telling Lies' Government-Surveillance Thriller Game Sets Release Date

    After more than two years in the works, “Telling Lies” — the investigative thriller from acclaimed game creator Sam Barlow — is ready to take the stage. The game, produced with and released by Annapurna Interactive, will be available next Friday, Aug. 23, via Steam and Apple’s Mac and iOS app stores. “Telling Lies” will [...]

  • David Messinger, Activision Blizzard

    Activision Blizzard Hires CAA Veteran David Messinger as CMO

    Activision Blizzard tapped David Messinger, a former 15-plus-year veteran of CAA, as its first corporate-wide chief marketing officer. Messinger, based in Santa Monica, reports to Coddy Johnson, Activision Blizzard’s president and COO. It’s the first time the video-game company has appointed a CMO who will oversee the global marketing operations across all of Activision Blizzard [...]

  • Editorial use onlyMandatory Credit: Photo by

    Oculus Co-Founder Nate Mitchell Is Leaving Facebook

    Oculus co-founder Nate Mitchell is leaving Facebook 7 years after launching the first Oculus Rift headset with a Kickstarter crowdfunding campaign. Mitchell announced the move on Reddit Tuesday, calling it “bittersweet news.” Mitchell said that he would take some time off to spend with his family. And while he committed to still being involved in [...]

  • nomadic seoul

    Nomadic Expands Abroad, Opens VR Center in Seoul

    Location-based virtual reality (VR) startup Nomadic is getting ready for an international expansion: The company will open its first VR center outside of the States in South Korea later this month. The new location, housed in the CGV Gangbyeon multiplex in Seoul, is being opened in partnership with theater chain CGV and technology provider CJ [...]

More From Our Brands

Access exclusive content