Facebook claims its new AI technology can automatically detect revenge porn

Image: Pictures Ltd./Corbis via Getty Images

Facebook is ramping up its fight against revenge porn.

On Friday, the company the launch of its new AI technology that it says can proactively detect “near-nude” images and video shared without consent. Facebook says that with this tool it can now automatically flag revenge porn before anyone even reports it.

Facebook previously relied on its users, often times the victims themselves, to notify the social network about this type of nonconsensual content found on the platform. This posed a problem.

“Often victims are afraid of retribution so they are reluctant to report the content themselves or are unaware the content has been shared,” says Facebook’s global head of safety Antigone Davis in a post.

Upon detection, specially-trained members of Facebook’s Community Operations team will review the flagged content.

“If the image or video violates our Community Standards, we will remove it, and in most cases we will also disable an account for sharing intimate content without permission,” writes Davis.

Facebook says this tool will work in tandem with its previous anti-revenge porn measure, the “non-consensual intimate image pilot” program. The pilot invites users to upload their intimate images, privately to the company, before they are posted anywhere. Facebook creates a “digital fingerprint” of that image which helps the company stop it from being shared on the platform. The program was when it was first announced but has since gained support from victims and support groups.

In addition to those two tools, Facebook also announced its victim-support hub with resources for its users on what to do and who to contact if they are a victim of revenge porn.

AI can be helpful when it comes to automatically detecting content. Facebook currently depends on artificial intelligence to be more proactive in other areas that affect its users’ safety, such as with its tool. However, even with this most recent moderation technology, it still a human being to make the ultimate call. 

Still, it’s a big step forward when a big tech company realizes the onus shouldn’t be on victims or its users to fix its own platform’s problems.