Since the very first moment I saw the iPhone X track a human face and display the results in real-time on an Animoji character, I've been waiting for the first great hack of this new iPhone feature.
Not only do we now have that first great demonstration, it's so impressive it's convinced me (the ultimate penny pincher) that I might have to buy this outrageously expensive smartphone for myself.
Award-winning effects creator Cory Strassburger recently got his hands on the iPhone X and decided to use ARKit and Unity to layer a real-time mask over his face, viewable on the screen of his iPhone X. The catch? The face is that of a hilariously expressive baby, designed by Strassburger, with shades and purple hair, or, as I like to call him, punk-rock baby Elvis.
Filming the demonstration using the camera on another smartphone, Strassburger spends about five minutes showing us the full range of the mask's ability to track facial ticks, grins, frowns, and even smoochy faces that all give the mask an oddly expressive functionality. You'll find it hard not to laugh watching this, but there's also something else happening here that is absolutely fascinating: we're looking at the future of possibly all video chat apps.
"Apple bought Faceshift a while back (facial capture software that tracked using depth data) and essentially made it mobile on the iPhone X, and through their ARKit API you can access the 52 different facially tracked motion groups which can drive your own characters blendshapes at 60fps," said Strassburger, referencing the work of his company Kite & Lightning, in a message accompanying the demonstration video.
"I'm interested in if it can be used for cheap and fast facial motion capture for our VR game which is over run by wild and crazy immortal babies who all want to express themselves."
- Don't Miss: iPhone X Users Create Animoji Karaoke, but for Future AR, This Is Bigger Than Singing Pigs & Unicorns
There are already video apps on the market for services like Facebook (via Masks in Facebook Live) that allow you to place a digital mask over your face when engaging in video chats, but without the detailed face-tracking ability of the iPhone X and its TrueDepth camera system.
However, while Facebook launched its Masks feature over a year ago, sharing Masks hasn't really taken off among most users. But thanks to the iPhone X's more detailed facial tracking — allowing users to transcend a mere mask and instead don a fully functional virtual face — there's a good chance that Apple's version of AR masks might begin to catch on.
Looking toward the future, the ability, as indicated by Strassburger's demonstration, to hold extended video chats wearing not just an Apple sanctioned mask, but a custom mask you've designed and/or purchased for yourself, is mind blowing.
As I mentioned in a previous article, these experiments are just the tip of the iceberg. The real AR fun will begin when someone either hacks the iPhone to track something other than the user's face or when Apple introduces next-gen, AR functionality itself.
For now, this experiment is a great start, and we're likely to see many more like it in the coming weeks and months.
"This is a quick first pass test and there's a bit more to be done before I hit the quality ceiling in regards to the captured data," says Strassburger. "Of course, if it were meant to be an AR gag, a lot lot more could be done to improve the visual rendering and lighting quality."
- Follow Next Reality on Facebook, Twitter, and YouTube
- Sign up for our new Next Reality newsletter
- Follow WonderHowTo on Facebook, Twitter, Pinterest, and Google+
Cover image via Cory Strassburger/YouTube
Comments
No Comments Exist
Be the first, drop a comment!