News: Facebook Adds Sketchfab, Body & Hand Tracking, Background Segmentation, & More to AR Studio

Facebook Adds Sketchfab, Body & Hand Tracking, Background Segmentation, & More to AR Studio

At its F8 developers conference on Tuesday, Facebook announced new tools and features coming in a new version of AR Studio, including the integration of 3D content aggregator Sketchfab's Download API.

Launched last month, the Download API enables developers to add 3D models directly from Sketchfab's extensive library of free content to their AR experiences.

Image by Sketchfab/YouTube

"We released our download API a month ago to integrate back in all the places that need 3D content, with the vision of providing a search bar for the 3D world," said Alban Denoyel, the co-founder and CEO of Sketchfab, in a blog post. "Partnering with Facebook is an important milestone on our path to make our download API the industry standard."

When Sketchfab announced its Download API last month, Facebook AR Studio was not one of the creator platforms the company noted that it was working with to integrate the tool. However, we did theorize that Facebook was a candidate to bring the functionality to the user side. Now, integrating the API on the creator side will make it easier for developers to import 3D content into their AR experiences.

Image via Facebook

In addition to Sketchfab, developers can access several new capabilities for their Facebook-related AR experiences, including several new targets for tracking. In addition to images, developers can now target hands and bodies (an advancement its research team previewed earlier this year), and track faces with higher fidelity.

The platform is also gaining semantic scene understanding, a dynamic that allows AR content to interact more meaningfully with real-world objects. We've seen AR content display a kind of contextual awareness before, such as Google's AR Stickers, but this enhanced awareness of physical objects (for example, making virtual steam rise from a real world cup when the cup is recognized in a scene) could give Facebook a leg up on its mobile peers.

Image via Facebook

Other new features bring Facebook's platform even with other AR options on the mobile market. AR Studio will also be capable of background segmentation, similar to what we've seen from Apple's Clips app and YouTube's stories feature. In addition, Camera AR will be capable of geofencing AR experiences to a specific location, mirroring capabilities seen in Snapchat.

Image via Facebook

The new version is "coming soon," but Facebook did not disclose a launch date in its announcement. We've reached out to a company spokesperson for clarification.

Last year, Facebook claimed that it was launching the first augmented reality platform. But before it could launch its AR Studio, Apple unveiled and launched ARKit. Nevertheless, with its new features, Facebook has now effectively caught up with its competitors and, in some areas, may be moving ahead of its peers in mobile AR.

Just updated your iPhone? You'll find new features for TV, Messages, News, and Shortcuts, as well as important bug fixes and security patches. Find out what's new and changed on your iPhone with the iOS 17.6 update.

Cover image via Facebook

Be the First to Comment

Share Your Thoughts

  • Hot
  • Latest