You will be redirected back to your article in seconds

Inside Netflix’s Plan to Boost Streaming Quality and Unclog the Internet (Exclusive)

The company is on a quest to re-encode its entire catalog

Over the past few months, Netflix has dared some of its employees in its Los Gatos offices with a special kind of challenge: Two TVs mounted side-by-side were playing the same TV show episode. One was coming straight from Netflix’s existing service, the other was based on a new bandwidth-saving technology that the company has been working on for four years. Anyone capable of pointing out the difference could win a bottle of champagne. But in the end, even eagle-eyed employees had to give up, and the prize went unclaimed.

Encouraged by these results, as well as months of additional testing, Netflix now has begun to embark on one of the biggest changes to its streaming technology since it launched its online video service in 2007. If all goes according to plan, the switch could help consumers get better-looking streams while also saving up to 20 percent of data — which is significant in North America, where Netflix usage single-handedly accounts for more than a third of all data consumed during peak times, and an even bigger deal in all those countries with relatively slow internet speeds that the company is looking to enter in 2016.

‘My Little Pony’ Is Not the Same as ‘The Avengers’

Netflix has been working on this new technology since 2011, when members of its video algorithms team realized that they had gotten it all wrong. Like practically everyone else in the online video world, Netflix had been preparing its video files for streaming based on the bandwidth available to consumers. Some Netflix subscribers were accessing the service with slow DSL connections, others had faster cable connections, and a lucky few were already online with super-fast fiber speeds.

Based on these use cases, Netflix’s video algorithms team had developed a number of quality levels, or recipes, as they’re called in the world of video encoding. Each video file on Netflix’s servers was being prepared with these same recipes to make multiple versions necessary to serve users at different speeds. At the lowest end was a file encoded with a bitrate of 235 kbps, which would work even on very slow connections, but also only deliver a resolution of 320 by 240 pixels. Somewhere in the middle was a 1750 kbps file for a resolution of 1280 by 720, and the best quality was a 5800 kbps version for a great-looking 1080p experience.

Netflix’s service has been dynamically delivering these versions based on a consumer’s bandwidth needs, which is why the quality of a stream occasionally shifts in the middle of a binge-watching session. But across its entire catalog of movies and TV shows, the company has been using the same rules — which didn’t really make sense. “You shouldn’t allocate the same amount of bits for ‘My Little Pony’ as for ‘The Avengers,’” explained Netflix video algorithms manager Anne Aaron.

To a computer in charge of encoding, ‘My Little Pony’ just isn’t that complex.
Courtesy of Hasbro / Netflix

That’s because a show like “My Little Pony” is not at all like the ‘Avengers’ — especially to a computer in charge of preparing media files for streaming. Animated shows like “My Little Pony” can be reproduced with relatively little data. An animated sky, for example, tends to be filled with the same shade of blue, and an animated pony that just stands around talking isn’t very complex to a computer either. The “Avengers” on the other hand is full of fast-paced action, which plays out in front of cityscapes and other environments with lots of visual details. And, of course, explosions, rubble, smoke, and lots of it. Or as video engineers like to call it: noise. “Noise is hard to encode,” said Aaron.

It’s a much more complex movie, at least from a purely visual perspective. This means that Netflix’s encoding servers have to work a lot harder to get it down to the same file sizes as they do with “My Little Pony,” leading to more compression and possibly more visible artifacts at low bitrates. That’s why in 2011, Netflix’s video algorithm engineers realized that they shouldn’t apply the same encoding rules to these two very different titles. “A one-size-fits-all model doesn’t give you the most optimal quality,” said Aaron.

Instead, they decided that each title should get its own set of rules. This allows the company to stream visually simple videos like “My Little Pony” in a 1080p resolution with a bitrate of just 1.5 Mbps. In other words: Even someone with a very slow broadband or mobile internet connection can watch the animated show in full HD quality under the new approach. Previously, the same consumer would have just been able to watch the show with a resolution of 720*480, and still used more data.

But Netflix’s new per title approach doesn’t just improve things for animated TV shows. Fans of other fare are also set to benefit, and see significant bandwidth savings when streaming the highest quality 1080p video. Netflix’s video algorithms team set up a test for Variety in its offices in Los Gatos last week, streaming two episodes of “Orange is the New Black” in 1080p on two TVs mounted side-by-side, similar to the tests the company has been doing with its employees. The images on both TVs looked virtually identical — but one streamed with 5800 kbps, using Netflix’s legacy encoding scheme, whereas the other displayed the show with 4640 kbps. The difference? 20 percent in bandwidth savings.

Each episode of a TV show can have different settings

Over the past couple of years, Netflix’s video algorithm engineers have worked on perfecting this more flexible approach towards video encoding. Initially, they assumed that they would just have to categorize all of their titles to develop new encoding rules for animated shows, action movies, slow indie dramas and so forth. But they quickly realized that no two indie movies are alike, and that there can even be considerable visual differences within a season of the same TV show. “Each episode could be very different,” said Aaron. The result is a true title-by-title approach, where every single movie and TV show episode gets its own encoding settings.

SEE MORE: Why Netflix Is Pushing ‘Breaking Bad,’ ‘House of Cards’ in 4K Ultra HD

In recent months, Netflix has been silently testing the new encoding scheme by sending out streams with new and improved bitrates and resolutions to randomly selected customers. The company did keep a close eye on completion rates and streaming duration, but it also wanted to make sure that all devices out in the field were able to deal with the new bitrates.

Separately, Netflix has been testing the new and improved videos in person both with its own employees as well as with a number of customers. And earlier this month, Netflix quietly added a first batch of videos encoded with the new per-title approach to its catalog. Beginning with some of the most popular videos, the company aims to have a thousand re-encoded titles in its catalog by the holidays, and complete the entire process by the end of Q1 2016.

Movies are re-encoded while you’re asleep

This whole endeavor couldn’t have been possible without some considerable technical advances. First of all, Netflix needed a good way to analyze the visual quality of a video file when encoded with different settings. The company doesn’t really talk about the size of its catalog, but it’s fair to assume that the entirety of its catalog, including all TV show episodes, movies and trailers across all of its markets around the world is in the six digits, and new titles are constantly added.

It’s impossible for Netflix’s employees to visually inspect every single one of these videos in all available resolutions and bitrates. That’s why Netflix partnered with researchers at the University of Southern California, the University of Nantes and the UT Austin to develop technology to automate this process.

But knowing which quality would be best for a video and actually redoing Netflix’s entire catalog are still two very different things. The company has been using two tricks to make this approach even more efficient and prepare itself for the massive re-encoding project: It cuts every title into numerous slices, making it possible for multiple servers to crunch away on it at the same time, significantly speeding up encoding.

Netflix has long used cloud computing instead of its own data centers to run its service, and the encoding is also done on Amazon’s web services. However, the sheer amount of computing power necessary for a redo of its entire catalog is still significant, even in a cloud world where capacity can be added on demand. That’s why Netflix is shifting around its encoding jobs on the server instances it rents from Amazon to make use of any idle time. Netflix’s Amazon servers may help you to binge on “Jessica Jones” in the evening, but at night, when everyone is asleep, those very same servers are busy re-encoding the company’s catalog.

New markets require better-looking videos at lower speeds

Netflix’s new per-title approach is good news for its subscribers, who now get to watch many of the service’s videos with a better quality while saving bandwidth at the same time. But it could be an even bigger deal for phone and cable companies and the internet at large.

Netflix is now responsible for 37 percent of all internet traffic going into people’s homes in North America during those peak times when everyone is in front of their TVs, according to recent data from Sandvine. Reducing that amount of data by up to 20 percent could help to alleviate internet bottlenecks. “We want to be good stewards of the internet,” said Aaron.

Netflix’s impact on the internet has been a bit of a touchy subject in the past. Cable companies like Comcast and Time Warner Cable wanted Netflix to reimburse them for sending so much traffic to their customers. Netflix argued that this would amount to double billing because those customers are already paying for their service. The internet companies disagreed, and a public stand-off temporarily led to Netflix speeds going down for many customers. In the end, Netflix gave in and agreed to pay up. But as the company is expanding around the world, similar issues are likely to pop up again. Anything that helps Netflix to slow down the growth of its traffic is likely going to make things a lot easier for the company.

SEE MORE: Netflix Wants the World: Can It Really Expand to 200 Countries in 2 Years?

What’s more, many of Netflix’s future markets have much slower internet speeds. Netflix has said that it wants to be in all countries around the world by the end of 2016. Expanding to India, Africa and the Middle East in many cases means taking a mobile-first approach, or dealing with much slower wired internet speeds. Both will benefit significantly from the new encoding scheme, which will allow Netflix to deliver better-looking videos at slower speeds.

Once Netflix’s video algorithms team is done with re-encoding the existing catalog, it already has a number of new challenges ahead. There is 4K, and there are a number of other ways to improve image quality, including higher frame rates and HDR.

And then there is another crazy idea that could require the company to re-encode the entire catalog all over again: After finding the best setting for each single video, Aaron’s team is now thinking about even encoding each scene of a movie or TV show with different settings to account for higher information density during fight scenes and lower demands during slow moments of introspection.

For Netflix employees, this could mean more chances to finally win that unclaimed champagne bottle.

More Digital

  • Chinese Tech Firm Huawei Seeks Content

    Beleaguered Chinese Tech Firm Huawei Seeks Content for Expansion Into Southeast Asia

    One of the most surprising first-time attendees at FilMart is Chinese tech giant Huawei, which has come to Hong Kong to acquire the video content it needs to support its strategy of expansion into Southeast Asia. The company is currently embroiled in a PR nightmare as it defends itself against accusations that its equipment could [...]

  • Canadian Animation Shop Guru Studio Makes

    Canadian Animation Shop Guru Studio Makes Moves in China

    Guru Studio, an award-winning Canadian animation studio, is exploring the possibility of opening an office in Hong Kong or mainland China. The Toronto-based company is keen to get a slice of China’s animation market, which has been growing quickly. Related Kevin Costner on Oscar Netflix Controversy: 'Movies Are for Theaters' Netflix Announces New Interactive Animated [...]

  • Netflix's Newest Interactive Animated Kids Show

    Netflix Announces New Interactive Animated Kids Show 'Battle Kitty'

    Netflix is doubling down on interactive content for kids. The streaming service announced a new interactive animated show on Tuesday dubbed “Battle Kitty” from Matt Layzell. The company didn’t share a premiere date for the show yet. “Netflix has given me the incredible opportunity to help change how young people see themselves,” said Layzell. “Like [...]

  • stream march madness 2019 online free

    NCAA Tournament: How to Stream March Madness Online

    The rankings are in, and the 68-team field is set. March Madness is officially upon us, and if you want to catch the action from home, from the office, or on-the-go, there are a couple ways to stream March Madness online for free. The first game kicks off today, and the tournament runs until April [...]

  • Google Stream Detailed, Named Stadia

    Google's Stadia Aims to Turn Any Screen Into a Powerful Gaming Console

    Stadia is Google’s game streaming service that is designed to let you play high-end games on most modern screens — be they phone, laptop, tablet, computer or TV — with very little download time and no loss of fidelity, the company announced Tuesday at the Game Developers Conference. The service is launching in 2019. Cost [...]

  • Amazon Focused on Building the Game

    Amazon Focused on Building the Game Industry's Digital Infrastructure

    While most of the discussion around Amazon’s presence in the game industry revolves around their scattershot game projects and rumored cloud gaming platform, the tech giant has been far more involved in building the backend infrastructure for major games. Companies like Square Enix and Zynga both use Amazon Games Tech, Amazon’s portfolio of services that [...]

  • patreon logo

    Patreon Launches Tiered Pricing for Creators, Introduces Rates for Micropayments

    Membership services platform Patreon is growing up: The company announced Tuesday that it is moving away from its one-size-fits-all pricing model for creators, introducing two new tiers in the process. In addition, Patreon is also standardizing its payment processing fees, and introducing a new dedicated rate for small-donor transactions. Patreon’s cut of the membership fees [...]

More From Our Brands

Access exclusive content