Header Banner
Next Reality Logo
Next Reality
Augmented & Mixed Reality News, Rumors & Dev Guides
nextreality.mark.png
Apple Snap AR Business Google Instagram | Facebook NFT HoloLens Magic Leap Hands-On Smartphone AR The Future of AR Next Reality 30 AR Glossary ARKit Dev 101 What Is AR? Mixed Reality HoloLens Dev 101 Augmented Reality Hololens How-Tos HoloLens v. Magic Leap v. Meta 2 VR v. AR v. MR

Forget Face Masks, Facebook Is Working on Full-Body AR Camera Effects

Jan 26, 2018 11:22 PM
Jan 26, 2018 11:24 PM
Illustration of human figures in various standing poses.

Facebook may have shamelessly copied Snapchat and its camera effects for faces (as well as its World Lenses), but it might beat its social media competitor to virtual body augmentation.

The AI Camera Team at Facebook has developed a method for detecting body poses and separating people in photos and videos from their backgrounds. In other words, the technology could make motion capture and green screen effects accessible to anyone with a Facebook app. Take that, Hollywood!

The research team also envisions the technology being applied to gesture controls for video games (sort of like Kinect for mobile devices). It could also be used to visually anonymize people to protect their privacy.

As with AR face masks, the body mask technology relies heavily on computer vision and artificial intelligence. However, while face masks require tracking and the detection of the mouth, eyes, and other facial features, the full-body version must be able to detect body movements, poses, and clothing, as well as have the ability to identify multiple people in a scene.

To accomplish this feat, the team used the Mask R-CNN framework for object detection and segmentation. On Monday, the team offered a detailed explanation of the technology via a blog post.

With the methodology in place, the team plans to work on "new model architectures" that can make the technology more efficient and lightweight for mobile devices.

Consumers have a seemingly insatiable appetite for AR front-facing effects from Facebook and Snapchat, so expanding such effects to include full body capture could open up compelling opportunities for creative expression. Moreover, the method's potential for gesture recognition could make it a standard for future AR interfaces.

And while this just research for now, and there's no indication as to when this might make it into an actual app, it's nevertheless an exciting development to watch.

Cover image via Facebook

You already know how to use your phone. With Gadget Hacks' newsletter, we'll show you how to master it. Each week, we explore features, hidden tools, and advanced settings that give you more control over iOS and Android than most users even know exists.

Sign up for Gadget Hacks Weekly and start unlocking your phone's full potential.

Related Articles

Comments

No Comments Exist

Be the first, drop a comment!