Facebook’s artificial intelligence can understand what’s in your photos

Facebook is working on artificial intelligence (AI) feature that can understand what’s going on in your photos, which in turn would help your Newsfeed perform better. This feature will allow it to answer questions about a photo, a feature aimed at helping blind people “see” images uploaded to the social network.

Facebook is working on AI that can tell what is in your photos

Speaking at Web Summit, Facebook’s CTO, Mike Schroepfer, detailed a new system the company has been developing to understand what’s happening in photos and translate that into natural language elements.

At Web Summit conference in Dublin, Ireland, Facebook’s Chief Technology Officer (CTO) Mike Schroepfer, explained the new system which FB been developing to understand what’s happening in photos and convey that into natural language elements.

“The beauty of AI systems is that we can get enough scale to deploy this to everyone on the planet,” said Schroepfer in a briefing for reporters.

Earlier this year, the social media giant unveiled a technology called “Memory Networks” which allows a machine to perform sophisticated questioning and answering. He showed an example where a computer was fed the synopsis of the “Lord of the Rings” and was able to answer questions about the story.

The goal is to help computers to “understand language more like a human would with context, instead of rote ones and zeroes memorization like a machine.”

One illustration displayed a photo in Newsfeed of a man skateboarding, which the computer deconstructed and explained in words to understand what was happening.

It broke it down into what it saw “a skateboard, a man, a trick, his skateboard” and constructed a description of what was happening.

Schroepfer said that “when the machine understands attributes of an image” it is “easy to understand the applications to Facebook.”

The other illustration using the same technology showed a prototype application that was targeted at visually impaired users. A photo is fed into the app, and you can ask questions using speech, which the AI will describe in words.

Another effort uses “predictive learning” to study images and make informed predictions of what will happen next, such as whether a stack of blocks is stable enough to stand or fall over.

The social networking site, which has more than 1.5 billion global users, has also sponsored efforts to build drones and laser networks to help reach billions more people who currently do not have access to Internet. Schroepfer said Facebook can use artificial intelligence to understand the huge amount of information users upload and predict which messages they will want to see.

“We need systems that can help us understand the world and help us filter it better,” he added.

Facebook is pursuing a world where it only displays to you things that you are interested in. The technology would allow the company to “do the best job of showing you what you want in your feed by understanding photos based on the pixels.”