Connect with us

Technology

Facebook’s Rosetta AI can read all the memes

Published

on

Facebook's new Rosetta AI can read memes.
Facebook’s new Rosetta AI can read memes.

Image: Sean Gallup/Getty Images

There are so many memes on Facebook and Instagram that the company has enlisted its artificial intelligence to help understand them.

In a blog post, Facebook developers say they have created a dedicated AI tool, called Rosetta, to read the text that appears in memes (and other images and video frames) that are shared to Facebook and Instagram.

At face value, understanding memes might not seem like the most important problem for AI to solve. But Facebook’s researchers point out that the technology, which is designed to recognize depicted text in a wide variety of languages, has many practical uses.

For example, Rosetta can read photographed text in menus and street signs, as well as words appearing on clothing and product labels. So while Rosetta isn’t dedicated to memes, their prevalence on Facebook and Instagram will undoubtedly make them a major use case, especially in Facebook’s detection of offensive material.

“Understanding the text that appears on images is important for improving experiences, such as a more relevant photo search or the incorporation of text into screen readers that make Facebook more accessible for the visually impaired,” Facebook explains, adding that reading text in images is important in identifying “inappropriate or harmful content and keep our community safe.”

According to Facebook, the system is able to process more than a billion images a day.

An illustration showing how Rosetta analyzes more than a billion images a day.

An illustration showing how Rosetta analyzes more than a billion images a day.

The process essentially boils down to two steps: Rosetta scans images for text, and then uses text recognition to identify what the text actually says. Once the text has been transcribed, the system interprets what the text could mean.

Through Rosetta, Facebook is able to improve its image search as well as the systems that determine the types of images that may appear in your News Feed. It also helps the company automatically detect and remove hate speech that it may not have been able to identify previously.

In the future, Facebook says it could apply the same technology toward understanding text that appears in video as well, though that requires a more complex system.

Https%3a%2f%2fblueprint api production.s3.amazonaws.com%2fuploads%2fvideo uploaders%2fdistribution thumb%2fimage%2f86590%2f95c72eed 1471 4d09 9af9 12ad921760b9

!function(f,b,e,v,n,t,s){if(f.fbq)return;n=f.fbq=function(){n.callMethod?
n.callMethod.apply(n,arguments):n.queue.push(arguments)};if(!f._fbq)f._fbq=n;
n.push=n;n.loaded=!0;n.version=’2.0′;n.queue=[];t=b.createElement(e);t.async=!0;
t.src=v;s=b.getElementsByTagName(e)[0];s.parentNode.insertBefore(t,s)}(window,
document,’script’,’https://connect.facebook.net/en_US/fbevents.js’);
fbq(‘init’, ‘1453039084979896’);
if (window.mashKit) {
mashKit.gdpr.trackerFactory(function() {
fbq(‘track’, “PageView”);
}).render();
}

Continue Reading
Advertisement Find your dream job

Trending