Netflix Replacing Star Ratings With Thumbs Ups and Thumbs Downs

Stranger Things
Courtesy of Netflix

Get ready to say goodbye to star ratings on Netflix: The company is getting ready to replace stars with Pandora-like thumbs ups and thumbs downs in the coming weeks.

Previously given star ratings will still be used to personalize the profiles of Netflix users, but the stars are disappearing from the interface altogether.

Netflix VP of product Todd Yellin told journalists on Thursday during a press briefing at the company’s headquarters in Los Gatos, Calif., that the company had tested the new thumbs up and down ratings with hundred of thousands of members in 2016. “We are addicted to the methodology of A/B testing,” Yellin said. The result was that thumbs got 200% more ratings than the traditional star-rating feature.

Netflix is also introducing a new percent-match feature that shows how good of a match any given show or movie is for an individual subscriber. For example, a show that should close to perfectly fit a user’s taste may get a 98% match. Shows that have less than a 50% match won’t display a match-rating, however.

Yellin said that the company completely relied on its users rating titles with stars when it began personalization some years ago. At one point, it had over 10 billion five-star ratings, and more than 50% of all members had rated more than 50 titles.

However, over time, Netflix realized that explicit star ratings were less relevant than other signals. Users would rate documentaries with 5 stars, and silly movies with just 3 stars, but still watch silly movies more often than those high-rated documentaries.

“We made ratings less important because the implicit signal of your behavior is more important,” Yellin said.


Filed Under:

Want to read more articles like this one? SUBSCRIBE TO VARIETY TODAY.
Post A Comment 76

Leave a Reply


Comments are moderated. They may be edited for clarity and reprinting in whole or in part in Variety publications.

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

  1. Andrew Starr says:

    Simply put. This new system sucks and I will be leaving Netflix because of it.

  2. cinephile says:

    As a long time (dvd AND stream) member for more than a decade, who is from San Jose, CA, and has rated several THOUSAND Netflix movies, I am disgusted at this simplistic & binary method which provides me no indication whatsoever about any movies. I believe I am poised to speak on this as such and have found the options that COULD have taken or SHOULD take place are/were this:
    1) Netflix asks their customers for their opinion on the ratings system before making a move
    2) Netflix could have two rating systems simultaneously (for those that argue algorithms exist(ed) both ways, then why not have both)
    3) Customers who rely on the 5 star method are now forced to migrate to other websites (which you kinda don’t want with today’s digital and fleeting attention span) to check out ratings to confirm or explore their watch list. I have never heard of a company that would want to their customers to LEAVE their website. Netflix is co-signing this so why not acknowledge it?
    4) Customers can join in Netflix’s (aka Yellin’s) flippant remark about being antiquated and just VOTE DOWN NETFLIX altogether and cancel their subscription(s). If I wanted to use this like a DATING system, I’d probably ALREADY be on a date. How do you mess up even “Netflix and Chill” and take it this far by comparing it to such. Netflix is not a dating app: newsflash! If so, it’s time to “see other people: because “it’s not YOU, it’s ME.”
    6) Movie watching truly involves COMMUNITY and CRAFT and while people do not necessarily DEPEND on what other people think (some really do rightfully so), they might quite like having the choice to see HOW others think and choose accordingly (or not)! Netflix could take a message from a non-“antiquated” biz book called Think Big Act Small: “Once you have decided to listen to your customers , you need to make sure you are responding to what they really want and know exactly what they are talking about.” (see most folks who watch movies like to READ, too). A Lot. Here’s hoping Netflix will read all that is being said about them on this topic. Which brings me to:
    7) Netflix could have a spot where you DIRECTLY provide feedback on the website. If customers are clueless sheeple and can only accommodate pushing up or down on the great big fum (side note: what could that say about the content then really?), surely they are not capable of communicating properly or have valid feedback and thoughts- so just do away (wait, has it ever BEEN there?) with customer service via e-mail, chat (oooooh, that has been inactive lately- so serendipitous!), and phone wait times.
    8) Netflix could fix the problem and admit: Ooops! We thought we knew better than you (paid subscribers) since we are so used to “recommending” things all the time, but clearly that is NOT the case and we see that now…what, with all the emotional backlash and posts and such. No one loves a company more than one which messes up badly and then rectifies the problem (swiftly). T minus ten.
    9) Those that do have time to waste (not those that are so [insert adjective here] to only vote up or down per the cool and “current” Netflix who is choosing to leave their most loyal customers in the dust) can commence going to their Netflix search bar, typing “netflix original”, and then downvoting all that low hanging fruit, because, hey, even if you dislike it just a smidge, it deserves to be thumbs down or nothing according them (and that would include their own content, right?) Dumb.
    and lastly, something I’ve always wondered,
    10) Netflix could actually FOCUS on featuring (and highlight) the HIGHEST RATINGS, MOST LIKED, and even NEWEST(not antiquated!) movies IN GENERAL instead of telling me what I want to watch. Please don’t tell me the up/down is what will solve this, because if that was the idea, I’d rather be able to REMOVE categories/banners, low rated movies, and crappy content from my sight . Period.

    Excuse any grammar, typos, etc as this topic is a little incendiary to me. Change what the stars say, make it a scale of 1 to 10 or only 4 stars, etc. but DO NOT tell movie lovers (your primary and first customers if you will recall) to resort to “Love It or Hate It” and that’s it. “What is food to one man may be a fierce poison to others”- Lucretius

    • WaltDittrich says:

      Great comments, Cinephile. I understand your frustration.
      I’m like you: I’ve been with Netflix since 2003, long before streaming started (in 2007). I’ve rated over 8000 titles.
      I’ve followed Netflix’ antics on sites like the now-defunct “Hacking Netflix”, so I understand how Netflix’ rating system works, and of course, how their recommendation engine, CineMatch, works (or worked). Netflix has always stated they had a 5 star scale, rather than ten, or half stars, because they found when they give people too many choices, they don’t rate as often. Switching to a Thumbs Up/Down approach matches that thought process: they get 200% MORE people to rate.
      This definitely leaves people like you and me out in the cold. We always rated. Without our ratings being seen, we feel jilted. Unfortunately, there are less than four million DVD subscribers now, while over 50 million streamers in the United States alone! This means that Netflix has to get those new customers to rate.
      I don’t stream much at all. This weekend brought me to this article because I wanted the streaming service to recommend something good to me, like the DVD side does. After reading this article, I see how important the % match system is. It’s like the red stars before. Most importantly, this article says if there is no percent match, then you’re less than 50% likely to enjoy the title you’re looking at. This is important! Now, you don’t have to waste your time (Other than it’s not a perfect system).
      The most important thing, to me, is that people do what you said: Thumbs Up equals LOVE something, Thumbs Down means you HATE something. Anything in between, just rate it on the DVD side, or not at ll (no in-between thumbs!).

      Netflix, via telephone, said they are NOT doing away with the stars for their DVD system. This is great news! Perhaps, they will find the thumbs don’t work, and will go back? Or, perhaps the thumbs will work better than WE think, and we’ll come to embrace them?

  3. enmukee says:

    Netflix just did everything they could to give Amazon Prime a boost. I am watching more Amazon Prime than I have ever done in the last 6 years now as I only have time to watch movies which a lot of other people have rated highly. I just do not have the time to watch their machine learning recommendations, besides my wife has messed up my Netflix profile with her tastes. Getting 200% more ratings means nothing if those ratings are meaningless. This entire practice of corporations catering to one’s tastes is what is causing people to be living in a bubble and is polarizing society. Let me show you news you will like, let me point you to movies you will like, it is crazy!

  4. Jman says:

    Exactly, first thing that came to mind was Amy Schumer’s horrible appearance and retched show. After all these years stars were fine and honestly paint a clearer picture of what the public thinks other than a “binary” thumbs up or down. First sign of Netflix “sold out”. Your not fooling anyone Netflix. I call it like I see it… and it stinks just like Schumer’s……

  5. Lisa says:

    I HATE IT SO MUCH!!! I don’t want netflix deciding what is a good match for me. They already put in their two cents for all their suggestions on the home page saying “Because you watched…” A movie to me isn’t just “good” or “not good” and I’m having a hard time finding something actually “good enough that I want to watch it” because of their shoddy matchmaking attempts that I didn’t ask for. If it doesn’t change back I’m leaving netflix.

  6. jim says:

    I agree with the comments; thumbs down on new system but maybe they are right and most people prefer a dumbed-down system. Without the stars I am no longer “locked in” to Netflix so maybe something good will come of this. I’ve already started reducing the cost so I can wean myself off Netflix completely (like I did with cable TV).

    An experiment: since their suggestion system does not work for me anyway and the thumbs don’t provide me with any useful information, at first I was going to protest by refusing to rate anything BUT a better idea may be to give a “thumbs DOWN” for everything; not representing the movies but for Netflix abandoning the stars system. If nothing else it may help mitigate some frustration each time I watch a movie. Maybe they will listen, but probably not. Give us the option to use stars even if they are not used in your algorithm!

  7. Jim Eubanks says:

    Thumbs down to the new system.

  8. blue glacia says:

    The new Netflix rating system eliminates the efficiency of choosing a movie and also for cataloging for later viewing with friends, because thumb up or down really means very little. What does “thumbs up” mean? A so-so movie, a nice movie or a really great movie? As a result, I find myself not watching Netflix altogether and gravitating toward amazon prime, as I don’t want to waste a lot of time finding out which movies are worth watching or re-watching.

  9. great, so now everything made by Netflix can all have the same thumbs down and be avoided by everyone.

  10. Nanny Mo says:

    Here’s Netflix’s comment line: Call them, be polite and tell them you want to have both systems. 1-866-579-7172. That thumbs up and down is too simplistic a system for you. Maybe they will listen.

  11. Nanny Mo says:

    Oh, crap. I used the stars to tell me what I had watch before. Three stars meant, I watched it and it was okay. Two meant don’t rent this again and 4 meant rent again, while 5 meant, loved it, rent it when you are sick in bed and need another winner. This new system will mess me up. Why can’t we have both? Ups and downs for suggestions from Netfilx and leave me my stars! Maybe it is time for Hulu.

  12. Joseph T Jones says:

    Horrible rating system for video. Thumbs up and down just doesn’t work with movies.

  13. Bill B. says:

    If you’ve been with them from the beginning, all they have ever done is reduce things for fans.

  14. JoeMcG says:

    Two points on this… 1) I will really miss the stars… I really enjoyed being able to rate the titles I’ve watch. Trouble is, most people have a tendency to “over-value” a star rating. I like a lot of films, but only just a handful earned 5 stars from me. Similarly, only a handful ever got 2 starts. To me a simple yes-no option is too strong. I won’t be using it. Which takes me to point 2… 2) I absolutely HATE these algorithms that decide what you should like based on what you’ve watched. I don’t care “what’s trending” or “because I watched…” How about you just show me “what’s new on Netflix”? Whatever happened to that option? I don’t want to be pigeon-holed based on my previous watching habits… I have broad tastes, and want to see all the options! How else can I be exposed to something new? This is also why I hate streaming music services. I don’t want some computer determining what they think I would like, and not show me the rest that’s out there. LET ME CHOOSE!

    • MrGrizzle says:

      Could not agree with you more! Give me the option to dislike something, don’t tell me what I may and may not like.

  15. MrGrizzle says:

    I’m not sure I like this. For example, I LOVE stand-up comedy but I don’t like Amy Schumer. So if I click thumbs down on Schumer, will that cause Netflix to think I don’t like Stand-up Comedy???

  16. OJ Obama says:

    The ratings are fake anyway. Just like imdb

  17. oseunth says:


  18. Anna says:

    The star system was one of the better things that Netflix had.

  19. Arch Stanton says:

    So this is the Amy Schumer effect?

  20. MovieGeek says:

    Not everything is black or white.
    Very bad idea

  21. Albin says:

    I use IMDB (not Netflix) to rate and read viewer ratings and the ten-point is useful. The ten point rating system forces some thought, and my detailed system actually squares up pretty well with the public: 1-4 Unwatchable, 5 Watchable concept, bad flick, 6 Watched through but meh, 7 Good one-time, 8 Watch again, 9 Fine Film, 10 World Classic. Thumbs are dumb.

  22. BrianDouglas says:

    My stars don’t matter anymore? Netflix doesn’t matter anymore. It worked fine. If it ain;t broke don’t fix it. I WILL GIVE netflix 2 months to make me enjoy it more – if it does not work BETTER than stars, I WILL UNSUBSCRIBE.

  23. Enzo says:

    spent time actually rating the shows and movies all for nothing. So now I guess I will just thumbs down everything I watch.

    • fofer says:

      Which part of “previously-given star rating will still be used to personalize the profiles of Netflix users” did you miss?

      • dcksckngckfg says:

        You’ve missed the point. What if IMDB had a thumbs-up/thumbs down rating system? How useful would that to be?

        With star ratings, rating the content was a public expression. Stars were ‘global’, thumbs up/down is entirely local/private.

        I’ve never bothered watching anything Netflix suggests. I don’t enjoy shows based on previous shows and movies I’ve watched, that’s asinine, I enjoy shows based on, primarily, quality. Thumbs up/down has nothing to do with quality and even if it did, you have to sit through it yourself to find out instead of having a general consensus.
        It’s a form of censorship, almost. Netflix users are no longer allowed, within Netflix itself, to publicly express the quality of the content.Even if it was still public, what do I care rather or not you ‘like or dislike’ a show or movie?
        The point is, for many people, probably most people, the star rating served an entirely different purpose which Netflix has now removed and replaced with.. nothing. Thanks Netflix. Not that the star ratings for most content was even accurate, but I’ve never come across any movie or series which had a 1-3 star rating and didn’t not suck.

    • Down rating a show doesn’t hurt Netflix, it hurts the producers, writers, and stars of the show. If Netflix sees a lot of down ratings, they’re less likely to license the content. That means the cast and crew don’t get paid. Netflix is still making money.

      If you’re worried about all that effort rating shows going to waste, fear not. There’s a multitude of tools out there to export your ratings. For example, I did a quick Google search and found one. I can’t vouch for its quality, but I wanted to provide a possible solution. Keep in mind that there’s several other mechanisms out there for doing this.

      You shouldn’t take this move personally, it’s meant for the greater good and it may actually improve their recommendation algorithms, though that may not be obvious at first glance.

      • Nanny Mo says:

        Then why can’t we have both? Stars for personal use and then I can up and down thumbs for, yes, more like this and no thanks, less like this. Why all or nothing?

  24. This will mix all the prior 3 star reviewed movies and shows with the 5 stars. You now either like it or not, you don’t get to choose “it was just ok.” We know what Netflix is doing and it’s a very bad idea. You would think they would add more stars to get a more accurate review. NO…they delete the stars all together.

    • They used stars to determine what a user was likely to watch. What they determined, through empirical evidence, is that the granularity of the stars wasn’t relevant. You’re watching 3 star shows just as much (if not more) than 5 star shows. Not only was the granularity not necessary, they got more feedback from users when the rating system was simpler. 3 fold the feedback, in fact (200% more).

      This wasn’t a decision made on a whim. It was throughly tested and vetted. YOU may think stars are better, but the evidence shows they’re not. I realize you don’t > like < it, but that doesn't make you right.

      • Matt says:

        I watch more 3 star shows because half of their catalog is crap…If they had more 5 star content I’d happily be watching it instead.

  25. Nathan T Howell says:

    I think this is in response to Amy Shumers not so hot response to her special, suck it up maybe wasn’t so funny.

  26. James Haylow says:

    Easier? Just how hard was it to click a star?

    • It’s not hard to click a star, it’s hard to know which star to click.

      What exactly am I rating with the stars?

      Do the stars represent the likelihood that I’ll watch it again? My enjoyment? The quality of the movie? Sometimes I find bad movies incredibly entertaining. The movie itself is a 2 but my enjoyment was a 5.

      This is just an example. There’s so much nuance to it. It’s not as trivial as you think. A like or dislike, however, is pretty simple. I enjoyed it or I didn’t.

      • cheshire cat says:

        the stars meant how much you liked it..they actually say that when you hover over them.
        2 = i didnt like it, 3 = liked it, 4 = really liked it, 5 = loved it.
        what it sounds like is people were using it exactly as netflix intended but then they decided they wanted it to represent how frequently you watched it which is not the same thing and entirely meaningless as a user of the star recommendations.

        i will watch fluff tv in the background on repeat but i will pay full attention and watch something good only once. if netflix dont understand that then i fear for their future because a service full of frequently watched mediocrity isnt going to have a lot of subscribers.

  27. nickbotic says:

    How about the option to choose how many episodes I want to see before being asked if I’m still watching, or better yet, the ability to turn that off completely! That, and the option to turn off the autoplay when a title is selected. Drives me nuts!

  28. N Prov says:

    Look what you Liberals did now. Netflix: You can’t fix Amy Schumer’s new specail like this

  29. Duh says:

    Instead of this stupid thumbs up/thumbs down, how about you give us the ability to rate (in a scale of 1-5 stars) seasons of shows independently? Why can’t I give the first few seasons of Dexter 5 stars, but the last season 1 star? I’m I just supposed to factor the show’s entirety into my ultimate rating?

  30. Matt says:

    Thumbs up/down ratings work well enough for gauging the public’s opinion (see: Rotten Tomatoes, IMDB) but when I know it influences my recommendations, I want to be able to express more nuance than yay or nay. I don’t think 1-5 ratings really do the job either, but this seems like a step backward.

    • Matt says:

      Sorry, not IMDB. They use a 1-10 rating system.

      • Terry Dio says:

        I’d love to be able to rate from 1-10 as it more clearly could define how much I liked it or not. Way too many lower quality movies there now and getting a bad one sent to me is just a huge waste of that mailing as far as I can see. Not a good decision.

  31. John doe says:

    Lame. This will simply allow netflix to hide the ratings on 1-star shit shows that they try to push on us in seemingly half the available genres. Boo neflix. Boooo.

    • Karmine Applesauce says:

      Yes, cause a 1-star show will get thumbs up? No this point is invalid and doesn’t make sense. The rating system went from rate 1 to 5 * which differs from person to person * to was it good or not? if people say its good, then its good.

      • Bryan Carter says:

        Are you posting a comment to an article you did not read? Your point does not make sense.

        Under the new system we will not see ratings for movies/shows but instead a percentage that predicts the likelihood it matches a user’s taste. Such a percentage is not the same as the percentage of thumbs up votes the move/show received.

  32. theintownwriter says:

    Hey, Netflix, do you think that the country has become so dumbed-down that your customers are incapable of evaluating *levels* of quality, and can only manage the black-and-white assessment of thumbs up or down?
    You insult us.
    One more nail in your coffin, from someone who’s been a subscriber since 2000. Amazon, Hulu and Acorn are singing a different song these days. One this soon-to-be-former Netflix customer appreciates more and more.

    • You’re taking this move awfully personally. Did you read the article?

      They did a study of the ratings system versus the like/dislike system. They found that the ratings didn’t matter in terms of gauging your likelihood of watching a show. People watched a 3 star shows just as much (if not more) than a 5 star shows. Not only was it not accurate for determining what you’d be interested in watching, but they got 3 times the engagement with the like/dislike system. That’s not a paltry number either, we’re talking millions more at scale.

      Speak for yourself when you say you’re insulted. Netflix did their homework, and I respect that.

      • michael says:

        Netflix’s justification is that silly movies that got 3-star ratings were watched more than documentaries that got 5-star ratings. I would counter that more people watch silly movies than documentaries regardless of the ratings, and that simple viewership numbers are not necessarily reflective of whether the viewers actually liked the movie or not.

  33. I’ve been shouting into the wind like a maniac that Netflix needs a feature to hide anything with 2 stars or fewer. Hopefully this is the next chapter in that custom experience. I like the idea of stopping a movie midway, clicking thumbs-down, and never seeing mention of it ever again.

  34. sadik kapadia says:

    I actually wrote their recommendation system (about 5 years ago). They are still using it today. Yes it is not perfect but they are unable to improve it (I currently have a factor of 3 times better performance). Primarily because Netflix misuse A/B tests and they don’t understand their recommendation system.
    Netflix also committed patent fraud with regards to the recommendation system. Expect to see more news later

    • Bryan Carter says:

      You wrote their ratings system? Netflix spent years and millions of dollars implementing their ratings and recommendation engine. They even had a contest at one point open to the public where they awarded $1 million for the most accurate algorithm that predict users movie preferences based on their ratings. If I recall they ended up using the winning algorithm because it was better than they had been able to develop at that point.

    • Simon Mahoney says:

      Sorry sir, but my pet rabbit and I created the rating system SIX years ago and we have 9 times better performance than the thing we ourselves created. You will all be hearing from my lawyer: “Mr.Tassles”.

  35. L C says:

    This is a mistake, the only reason this kinda “works” for rottentomatoes is because the people who rate are movie critics who rarely are unfair with the movies/shows, but i can’t trust casual netflix users to use this system on shows like Stranger Things any Marvel Netflix show

    • G M says:

      Only people that rate things the same way as you are relevant to you. If you and I rate 99 movies the same, then when I rate movie 100 before you, it is a safe bet that netflix can tell you that you will enjoy that movie the same as I did. But If you and I only rate 25 movies the same and 74 movies differently, how I rate movie 100 is probably going to be irrelevant to predicting how you will enjoy movie 100.

      tldr: Only the people who rate movies the same way you do are used to predict how you will like movies you haven’t seen.

      • Valfreyja says:

        Except you yourself just invalidated what you said. Rotten Tomatoes is a REVIEW site in which that sort of professionalism is a requirement for sound results. The netflix rating system ISN’T a review, it’s a suggestion platform designed to push content towards average users.

    • What if dumb-down culture means more corporate profits though?

  36. You know, there’s something to be said about specificity. Changing it to thumbs up or down. It’s such a ‘herd’ mentality. BOO! We HATE IT! Or, YAAY, WE LOVE IT!! Some things are more complicated than that. Like art. And film. Of course people LOVED the thumbs up/down. Less thinking to do. What’s next? A ‘grunt’ button if you liked it. A ‘growl’ button if it made you mad? Jesus. It takes thought to review films. I’m an actor. The least you can do is use one minute of brain power to decided what percent of four stars any given movie deserves. And some films are more than like or hate. Get it together, society. Art. Film. Stage- it all requires your input. For all you get out of a two hour film, the least you can do- is reflect on it for thirty goddamned seconds.
    *end rant, steps off soap box. Shoves it under arm, strides off to another audition- head high*

  37. Scott A says:

    This won’t help Amy Schumer and her very low rating on her recent “comedy”

    • Simon Mahoney says:

      That was comedy? I thought it was an art piece about the animal cruelty surrounding the cattle industry. I mean, she kinda waddled on stage in leather and died painfully and slowly for an hour. Guess I read it wrong.

  38. Netflix ‘recommendations’ for me have been so far off the mark I don’t see how this could possibly help.

  39. jtbuzz says:

    First IMDB screws the end-user, now Netflix. I vote thumbs down on this. They will be calling this Qwikster 2.

  40. The article didn’t mention this, but please tell me they are going to grandfather/convert all of the star ratings into thumbs up and thumbs down automatically. I have literally already rated thousands and thousands of titles. Please tell me 3 stars and up will convert into thumbs up and 2 stars and lower will convert to thumbs down. Seriously, please tell me that.

    • Ryan says:

      > Previously-given star rating will still be used to personalize the profiles of Netflix users, but the stars are disappearing from the interface altogether.

      They won’t be “converted” to thumbs up & thumbs down, but they will still have influence on your recommendations.

  41. Geri says:

    Pandora-like? Don’t you mean TiVo????

  42. lindsey says:

    “pandora-like”? ever heard of streaming video site YouTube? what a rube

    • Jake says:

      You’re thinking about this way too much.

      • nerdrage says:

        Actually I changed my mind lol! I think Netflix is trying to force people to be honest in their ratings. Instead of rating something like The Crown five stars because you’re supposed to like it, it’ll see that (for me), I only watched the first episode, so any rating I give will be less honest than the honesty I displayed by not watching further.

        One episode should be about a two star. If you didn’t get thru even that one episode, one star. Three/four/five depends on how fast you burned thru. Binge watching = five star, more sane pace = four star, finally slogged thru after weeks = three star.

        Netflix doesn’t even need to get a thumbs up/down, since they can already observe behavior, which is the most truthful measure. They probably don’t want people to revolt by taking away inputs entirely. I wonder if they’ll just ignore the thumbs up/down thing entirely.

More Digital News from Variety