YouTube's Algorithm: What's The Problem With It?

As a content sharing platform, YouTube has rules that its users must follow. Videos have to follow YouTube’s Community Guidelines , which cover everything from spam and external links to misinformation and hate speech. YouTube is a massive platform, though, with 2 billion monthly active users and more than 500 hours of content added every minute. It’s not surprising if some videos fall through the cracks. But it’s not just a few videos that violate YouTube’s policies and stay on the platform. In the last several years, it’s become a huge problem for the network. The YouTube algorithm has even been found to recommend videos that go against the guidelines. So what’s up? Why does YouTube’s algorithm recommend videos that violate its own policies? In July 2021, YouTube’s algorithm made headlines when the company Mozilla released a report stating that it appeared to recommend videos that aren’t in line with YouTube’s guidelines. Mozilla is the company that makes the Firefox web browser. In 2020, it decided to conduct a study on the YouTube algorithm. YouTube has been criticized in the past for its lack of transparency regarding its recommendations. The Mozilla study was the largest investigation ever into YouTube’s algorithm. The data for the study came from thousands of YouTube users who agreed to use an open source browser extension called RegretsReporter. The extension tracked the videos the users watched and asked whether they regretted watching each video. The results from the study show some serious issues with YouTube’s recommendation algorithm. Videos with misinformation were the biggest issue The study took place between July 2020 and May 2021. Throughout that time, 37,380 users flagged 3,362 videos as regrettable. The researchers watched these flagged videos to check them against YouTube’s Community Guidelines. They found 12% as videos that should not be on YouTube or should not be recommended by the algorithm. The most frequently found issues with the reported videos were: One important insight from the study is that 71% of the regretted videos were recommended by YouTube. Recommended videos were 40% more likely to be regretted than searched-for videos. The researchers also noted that in 44% of cases, they had data about videos a volunteer watched before reporting regret. The recommendation was completely unrelated to previous videos the volunteer watched. That means that almost half of the time, YouTube is recommending videos that have nothing to do with a user’s watch history and that users don’t want to watch. If these recommended videos are not related and users regret watching them, why is YouTube’s algorithm suggesting them? The Mozilla study also found that recommended and then regretted videos were getting 70% more views per day than other videos the volunteers watched. When using their figures to determine the harm YouTube’s algorithm does online, the Mozilla researchers said it’s complicated. For them, many of the reported videos fall into a “borderline content” category. Those come close to violating the Community Guidelines without actually doing so. One final insight from the Mozilla study that raises eyebrows is that the rate of regrettable videos was 60% higher in countries that do not have English as a primary language. The researchers attribute this glaring difference to the fact that YouTube’s algorithm trains on primarily English-language videos. Mozilla isn’t the only organization to carry out an investigation of YouTube’s algorithm. Several other studies have been done in recent years, including: What sets the Mozilla study apart from other investigations is its crowdsource approach and size. The company was able to capture hard figures related to YouTube’s recommendation algorithm. The Mozilla study was eye-opening and reinforced questions that many critics have been asking of YouTube for years. The investigation ended in May 2021 and the report was released two months later. Has YouTube made any progress since then? Not so much. Although the platform has made more efforts at transparency and stopping the spread of misinformation, experts say more can be done. YouTube did announce in April 2021 that it now uses a “violative view rate” metric showing how many videos of every 10,000 uploaded violate YouTube’s rules. The numbers were down to 18 per 10,000 at the end of 2020. YouTube’s executives herald this as progress in the fight against inappropriate content on the platform. But as Rebecca Heilweil from Vox points out , YouTube’s internal moderators decide what goes against the Community Guidelines, not independent auditors. In October 2021, YouTube started adding content policies related to COVID-19 and vaccine misinformation — almost 2 years after the start of the pandemic. A CNN Business report noted that anti-vax conspiracy theorists Dr. Sherri Tenpenny and Dr. Joseph Mercola were only banned on YouTube shortly before the new COVID-19 content rules went into effect. When it comes to transparency and fact-checking, YouTube still has some work to do. In a January 2022 open letter to YouTube’s chief executive, Susan Wojcicki, more than 80 fact-checking organizations claimed YouTube is a conduit of worldwide disinformation and misinformation. The fact-check groups requested YouTube make four important changes: The signers of the letter came from more than 40 countries across different types of organizations, from charities and foundations to privately funded groups. YouTube’s algorithm is supposed to flag inappropriate content automatically, and human moderators review the flags and decide whether to take down the video. Users can also report videos they believe violate YouTube’s policies. Getting flagged isn’t the same as getting removed, as a moderator still has to review the video. Takedowns happen on a case-by-case basis and often one report isn’t enough to get a video taken down unless the content is particularly bad. Channels that get videos taken down get a strike, and three strikes within a 90-day period is cause to delete the channel entirely. During the fourth quarter of 2021, about 3.75 million videos were removed from YouTube. Only 300,000 of those videos were taken down via flags from non-automated flagging systems. The top reason for video removal during this quarter was child safety, which were 32% of removed videos. The countries with the highest number of removals were India, the United States, Indonesia, and Brazil. Despite the removal of all the videos, critics still say YouTube could do more to correct its algorithm and promote transparency. YouTube could do with more transparency when it comes to moderation. Tightening rules and policies is all well and good. But if it never submits to an external audit, it’s hard to have a lot of confidence. The Mozilla researchers had some recommendations based on their findings. They think YouTube should: YouTube does publish quarterly transparency reports and released its first copyright transparency report at the end of 2021. They don’t provide a lot of information about the recommendation algorithm, however. The responsibility doesn’t fall entirely on YouTube, either. Policymakers can introduce laws that force platforms like YouTube to be more transparent about the AI systems they use. YouTube has made efforts at greater transparency in the last few years. There’s still more to be done, though. Will YouTube ever heed its critics and start using independent, third-party reviews of its practices? Perhaps, but it may not happen for a long time.