Inside Netflix’s Plan to Boost Streaming Quality and Unclog the Internet (Exclusive)

The company is on a quest to re-encode its entire catalog

Over the past few months, Netflix has dared some of its employees in its Los Gatos offices with a special kind of challenge: Two TVs mounted side-by-side were playing the same TV show episode. One was coming straight from Netflix’s existing service, the other was based on a new bandwidth-saving technology that the company has been working on for four years. Anyone capable of pointing out the difference could win a bottle of champagne. But in the end, even eagle-eyed employees had to give up, and the prize went unclaimed.

Encouraged by these results, as well as months of additional testing, Netflix now has begun to embark on one of the biggest changes to its streaming technology since it launched its online video service in 2007. If all goes according to plan, the switch could help consumers get better-looking streams while also saving up to 20 percent of data — which is significant in North America, where Netflix usage single-handedly accounts for more than a third of all data consumed during peak times, and an even bigger deal in all those countries with relatively slow internet speeds that the company is looking to enter in 2016.

‘My Little Pony’ Is Not the Same as ‘The Avengers’

Netflix has been working on this new technology since 2011, when members of its video algorithms team realized that they had gotten it all wrong. Like practically everyone else in the online video world, Netflix had been preparing its video files for streaming based on the bandwidth available to consumers. Some Netflix subscribers were accessing the service with slow DSL connections, others had faster cable connections, and a lucky few were already online with super-fast fiber speeds.

Based on these use cases, Netflix’s video algorithms team had developed a number of quality levels, or recipes, as they’re called in the world of video encoding. Each video file on Netflix’s servers was being prepared with these same recipes to make multiple versions necessary to serve users at different speeds. At the lowest end was a file encoded with a bitrate of 235 kbps, which would work even on very slow connections, but also only deliver a resolution of 320 by 240 pixels. Somewhere in the middle was a 1750 kbps file for a resolution of 1280 by 720, and the best quality was a 5800 kbps version for a great-looking 1080p experience.

Netflix’s service has been dynamically delivering these versions based on a consumer’s bandwidth needs, which is why the quality of a stream occasionally shifts in the middle of a binge-watching session. But across its entire catalog of movies and TV shows, the company has been using the same rules — which didn’t really make sense. “You shouldn’t allocate the same amount of bits for ‘My Little Pony’ as for ‘The Avengers,’” explained Netflix video algorithms manager Anne Aaron.

To a computer in charge of encoding, ‘My Little Pony’ just isn’t that complex.
Courtesy of Hasbro / Netflix

That’s because a show like “My Little Pony” is not at all like the ‘Avengers’ — especially to a computer in charge of preparing media files for streaming. Animated shows like “My Little Pony” can be reproduced with relatively little data. An animated sky, for example, tends to be filled with the same shade of blue, and an animated pony that just stands around talking isn’t very complex to a computer either. The “Avengers” on the other hand is full of fast-paced action, which plays out in front of cityscapes and other environments with lots of visual details. And, of course, explosions, rubble, smoke, and lots of it. Or as video engineers like to call it: noise. “Noise is hard to encode,” said Aaron.

It’s a much more complex movie, at least from a purely visual perspective. This means that Netflix’s encoding servers have to work a lot harder to get it down to the same file sizes as they do with “My Little Pony,” leading to more compression and possibly more visible artifacts at low bitrates. That’s why in 2011, Netflix’s video algorithm engineers realized that they shouldn’t apply the same encoding rules to these two very different titles. “A one-size-fits-all model doesn’t give you the most optimal quality,” said Aaron.

Instead, they decided that each title should get its own set of rules. This allows the company to stream visually simple videos like “My Little Pony” in a 1080p resolution with a bitrate of just 1.5 Mbps. In other words: Even someone with a very slow broadband or mobile internet connection can watch the animated show in full HD quality under the new approach. Previously, the same consumer would have just been able to watch the show with a resolution of 720*480, and still used more data.

But Netflix’s new per title approach doesn’t just improve things for animated TV shows. Fans of other fare are also set to benefit, and see significant bandwidth savings when streaming the highest quality 1080p video. Netflix’s video algorithms team set up a test for Variety in its offices in Los Gatos last week, streaming two episodes of “Orange is the New Black” in 1080p on two TVs mounted side-by-side, similar to the tests the company has been doing with its employees. The images on both TVs looked virtually identical — but one streamed with 5800 kbps, using Netflix’s legacy encoding scheme, whereas the other displayed the show with 4640 kbps. The difference? 20 percent in bandwidth savings.

Each episode of a TV show can have different settings

Over the past couple of years, Netflix’s video algorithm engineers have worked on perfecting this more flexible approach towards video encoding. Initially, they assumed that they would just have to categorize all of their titles to develop new encoding rules for animated shows, action movies, slow indie dramas and so forth. But they quickly realized that no two indie movies are alike, and that there can even be considerable visual differences within a season of the same TV show. “Each episode could be very different,” said Aaron. The result is a true title-by-title approach, where every single movie and TV show episode gets its own encoding settings.

SEE MORE: Why Netflix Is Pushing ‘Breaking Bad,’ ‘House of Cards’ in 4K Ultra HD

In recent months, Netflix has been silently testing the new encoding scheme by sending out streams with new and improved bitrates and resolutions to randomly selected customers. The company did keep a close eye on completion rates and streaming duration, but it also wanted to make sure that all devices out in the field were able to deal with the new bitrates.

Separately, Netflix has been testing the new and improved videos in person both with its own employees as well as with a number of customers. And earlier this month, Netflix quietly added a first batch of videos encoded with the new per-title approach to its catalog. Beginning with some of the most popular videos, the company aims to have a thousand re-encoded titles in its catalog by the holidays, and complete the entire process by the end of Q1 2016.

Movies are re-encoded while you’re asleep

This whole endeavor couldn’t have been possible without some considerable technical advances. First of all, Netflix needed a good way to analyze the visual quality of a video file when encoded with different settings. The company doesn’t really talk about the size of its catalog, but it’s fair to assume that the entirety of its catalog, including all TV show episodes, movies and trailers across all of its markets around the world is in the six digits, and new titles are constantly added.

It’s impossible for Netflix’s employees to visually inspect every single one of these videos in all available resolutions and bitrates. That’s why Netflix partnered with researchers at the University of Southern California, the University of Nantes and the UT Austin to develop technology to automate this process.

But knowing which quality would be best for a video and actually redoing Netflix’s entire catalog are still two very different things. The company has been using two tricks to make this approach even more efficient and prepare itself for the massive re-encoding project: It cuts every title into numerous slices, making it possible for multiple servers to crunch away on it at the same time, significantly speeding up encoding.

Netflix has long used cloud computing instead of its own data centers to run its service, and the encoding is also done on Amazon’s web services. However, the sheer amount of computing power necessary for a redo of its entire catalog is still significant, even in a cloud world where capacity can be added on demand. That’s why Netflix is shifting around its encoding jobs on the server instances it rents from Amazon to make use of any idle time. Netflix’s Amazon servers may help you to binge on “Jessica Jones” in the evening, but at night, when everyone is asleep, those very same servers are busy re-encoding the company’s catalog.

New markets require better-looking videos at lower speeds

Netflix’s new per-title approach is good news for its subscribers, who now get to watch many of the service’s videos with a better quality while saving bandwidth at the same time. But it could be an even bigger deal for phone and cable companies and the internet at large.

Netflix is now responsible for 37 percent of all internet traffic going into people’s homes in North America during those peak times when everyone is in front of their TVs, according to recent data from Sandvine. Reducing that amount of data by up to 20 percent could help to alleviate internet bottlenecks. “We want to be good stewards of the internet,” said Aaron.

Netflix’s impact on the internet has been a bit of a touchy subject in the past. Cable companies like Comcast and Time Warner Cable wanted Netflix to reimburse them for sending so much traffic to their customers. Netflix argued that this would amount to double billing because those customers are already paying for their service. The internet companies disagreed, and a public stand-off temporarily led to Netflix speeds going down for many customers. In the end, Netflix gave in and agreed to pay up. But as the company is expanding around the world, similar issues are likely to pop up again. Anything that helps Netflix to slow down the growth of its traffic is likely going to make things a lot easier for the company.

SEE MORE: Netflix Wants the World: Can It Really Expand to 200 Countries in 2 Years?

What’s more, many of Netflix’s future markets have much slower internet speeds. Netflix has said that it wants to be in all countries around the world by the end of 2016. Expanding to India, Africa and the Middle East in many cases means taking a mobile-first approach, or dealing with much slower wired internet speeds. Both will benefit significantly from the new encoding scheme, which will allow Netflix to deliver better-looking videos at slower speeds.

Once Netflix’s video algorithms team is done with re-encoding the existing catalog, it already has a number of new challenges ahead. There is 4K, and there are a number of other ways to improve image quality, including higher frame rates and HDR.

And then there is another crazy idea that could require the company to re-encode the entire catalog all over again: After finding the best setting for each single video, Aaron’s team is now thinking about even encoding each scene of a movie or TV show with different settings to account for higher information density during fight scenes and lower demands during slow moments of introspection.

For Netflix employees, this could mean more chances to finally win that unclaimed champagne bottle.

More Digital

  • Jingle Punks Jingle Player

    Jingle Punks at 10: How the Production Music Platform's Player Works

    Though its primary function is creative, Jingle Punks is built on a foundation of technology and administration. The patented Jingle Player that lets customers search for music using pop culture terms is both intuitive and efficient. Typing in “Reservoir Dogs” or “Starbucks” generates suggestions. Queries are monitored “so if there isn’t an exact match, we’ll suggest [...]

  • Jill Goldfarb - Jukin Media

    Jukin Hires TV Veteran Jill Goldfarb as VP of Linear Programming

    Jukin Media, which specializes in licensing user-generated viral videos, hired Jill Goldfarb as VP of linear programming. Goldfarb’s former tours of duty include serving as Discovery Channel’s VP of programming and as VP of program planning and scheduling at ABC Family/Fox Family Channel. Most recently, for the past five years she worked as an independent [...]

  • Imax Is Exiting the VR Space,

    Imax Is Shutting Down Its VR Business, Closing Remaining Three VR Centers in Q1

    Imax is making its exit from virtual reality (VR) official: The company notified shareholders with a SEC filing Thursday that it will close down its remaining three VR centers, and write off “certain VR content investments.” A company spokesperson confirmed the planned closures and shared the following statement with Variety: “With the launch of the [...]

  • Apple Music Connect Is Being Discontinued

    Apple Music Phases Out Connect Social Feed

    Apple Music has notified artists that it will be phasing out its Connect social feed. Artists won’t be able to post to Connect anymore effective immediately, and their existing posts will be removed by next May, according to an email sent to artists that was first published by 9to5Mac Thursday. “Today we’re streamlining music discovery [...]

  • YouTube logo

    YouTube Warns Creators They May See Subscriber Count Drops Amid Spam-Account Purge

    YouTube is enacting a broad purge of spam accounts over the next two days, and it’s warning creators they could see a big drop in subscribers as a result. According to YouTube, on Dec. 13-14, creators “may see a noticeable decrease” in subscriber counts. The Google-owned video service regularly works to verify the legitimacy of [...]

  • Ninja to Call 'Thursday Night Football'

    Ninja to Call NFL 'Thursday Night Football' Game on Twitch Free Live-Stream

    Tyler “Ninja” Blevins is Twitch’s most popular live-streaming gamer. Now he’s taking his play-by-play skills to the NFL — although it remains to be seen how many of his millions of followers are actually interested in his commentary on gridiron action. This season, Amazon-owned Twitch has provided free live-streaming feeds of the NFL’s “Thursday Night [...]

  • Amazon Starts Selling Chromecast Again, YouTube

    Amazon Starts Selling Google’s Chromecast Again

    Amazon is once again selling Google’s Chromecast streaming adapter, three years after yanking the product from its website. The e-commerce giant began listing the third-generation model of Google’s Chromecast streaming stick, as well as the 4K-capable Chromecast Ultra, on its website this week. The move could be a first step toward a more comprehensive business [...]

More From Our Brands

Access exclusive content