Snap Inc. Launches Local Lenses In London

Snapchat has just launched Local Lenses, a new feature that is one of the earliest examples of large-scale collaborative augmented reality (AR).

The dream of graffiti artists around the world has become a reality – now you can paint the central street of London, Carnaby Street, with any paints. Fortunately, these arts are easy to remove. It’s become possible thanks to the newly launched just launched Local Lenses by Snapchat.

This technology was first announced in June, and then promised to turn entire neighborhoods into digital canvases. It is now running in a small area around London’s Carnaby Street.

This first “local lens” is called City Painter. The idea is simple – using the Snapchat camera to view Carnaby Street, City Painter lets you spray huge fountains of red and blue paint over Carnaby’s shops, decorating the walls with pre-made murals. Anyone on the street can see others spray paint, and users can fight to cover stores with their own flowers.

Local Lenses is the latest app from the London-based camera development team Snap, which also created Landmarkers. Landmarkers is the company’s first large-scale augmented reality project that allows people to place user-created lenses on famous landmarks – like the Eiffel Tower shooting rainbows and cute characters appearing over Buckingham Palace.

Local Lenses is a more complex project, says Qi Pan, senior manager of engineering research at Snap London. There was much less data to work with – 3D data of famous public landmarks was easy to obtain, while Carnaby Street is completely pedestrianized, where users can shoot with their phone from anywhere without fear of movement. “People didn’t take pictures of every part of the street, simply because there is usually nothing interesting in the whole street,” he explains.

Another challenge is the huge size of the space. Landmarkers worked with users aiming the camera at one fixed landmark, usually from one point, to form a postcard-like image. City Painter, on the other hand, uses 3D rendering of the entire Carnaby Street so that users could draw on it from anywhere. The company had to extract visual data from several sources, including analysis of publicly shared images of Carnaby Street and its own 360-degree camera shots.

The second new addition to City Painter is the ability to share experiences. At Landmarkers, everyone was limited to their own augmented reality – what you saw on your phone was different than what everyone else saw. Local Lenses, on the other hand, is a one-stop-shop.

“We have one common reality,” says Pan. “And when you do something with this world, someone else can see this result almost instantly. These changes will also persist if everyone leaves the experience and new people appear the next day – they can still see the space that has been changed by you and others. “

Local Lenses is Snap’s latest AR project. Snap claims to have created over a million lenses using Lens Studio, a publicly available tool for creating new AR experiences. Snapchat users have played with lenses billions of times, and over 75% of Snap’s daily active users interact with AR every day. All of this activity, Pan explains, is training the Snapchat camera. “We want the Snapchat camera to really understand what it sees in the world.”

As the camera improves, its use will grow – from advertising to video games, live performances, and navigation. City Painter is a prime example of the development of the “mirror world” – the idea that every street, building, and room in the city will have a digital twin in the mirror world. Virtual experiences can be built around entire neighborhoods or entire digital versions of London, for example.

“We are planning to do new things that could not have been done before, such as AR tours of the districts led by famous people who lived here. You can leave notes, photos and videos for your friends to watch later, or play exciting AR games with your friends, changing the digital space, ” says Pan.

Stay tuned with Software Focus!