Google Lens gets useful new tipping, translating features

Image: raymond wong / mashable

A whole bunch of cool and useful new features are coming to Google Lens.

At this year’s I/O developer conference, Aparna Chennapragada, Google’s vice president of and general manager for Camera and AR products, shared several ways in which Google Lens can be used to help better understand the world around us.

In 2017, Google Lens was introduced as a feature within Google’s camera app and Google Photos with limited machine learning and AI capabilities. At launch, Lens could use image recognition to detect simple things like landmarks, phone numbers, and addresses within photos.

But later this year, Google Lens will get a lot more powerful. For example, you’ll be able to point Lens at a restaurant menu and its AI will automatically highlight popular items and let you pull up photos of them. (You know you already search for this stuff — don’t deny it.) Clearly, this new feature is the latest shot aimed at taking down Yelp. 

In another demo, Google showed off Lens’ new tip-calculating feature. Simply point the Lens camera at a receipt and it’ll automatically calculate the tip. For someone who hates opening up the calculator app to figure out how much to tip, this feature will be very useful.

Lens will also bring recipes to life. Pointing the camera at a Bon Appetit article featuring a recipe, for example, will bring up a list of actions to do to make the dish as well as a link to purchase ingredients.

Perhaps the most impressive new Lens feature is Google Go. Aim the camera at any text, such as a sign, and Lens will use the Google Assistant to read the text out loud, highlighting the words as it goes. 

Even more useful is a built-in translation feature, which translates the text and overlays it on top of the original image in real-time. It then reads the translation out loud. Go leverages all of the company’s core AI technologies — translation, image recognition, and a voice assistant — and puts it into a single product to create a “more helpful Google for everyone,” Google CEO Sundar Pichai said.