Report: Facebook auto-generates extremist videos

Image: Justin Sullivan / getty

Facebook has an automation problem. 

A confidential whistleblower complaint filed to the SEC and obtained by the Associated Press claims that the social network has been generating extremist videos, pages, and content by default. The content in question, which reportedly was manufactured entirely by Facebook independent of any specific human, ranges from white supremacist pages to pages for Al-Qaida.

Yeah, it’s bad. 

According to the AP, Facebook’s tools “[scrape] employment information from user’s pages to create business pages.” When those users’ pages contain extremist content, like, for example, pictures of suicide vests or mushroom clouds detonating in cities next to the words “The Islamic State” (two real examples), those images can make their way into Facebook autogenerated content.  

Image: screenshot / facebook

We reached out to Facebook for comment, but received no response as of press time. That doesn’t mean, however, that the company hasn’t just recently spoken to the power of its AI systems. 

“AI powers a wide range of products at Facebook,” the company explained in a F8 blog post. “In recent years, this has included our work to proactively detect content that violates our policies. To help us catch more of this problematic content, we’re working to make sure our AI systems can understand content with as little supervision as possible.”

In allegedly assisting in the creation of propaganda material, Facebook’s algorithms have demonstrated that they still have a long way to go. 

Image: Justin Sullivan / getty

Mark Zuckerberg has often fallen back on the excuse that one day, perhaps soon, AI will be able to successfully moderate content on his platform. Today’s AP report, however, shows how quickly unsupervised programs can go awry.