News: Facebook Plans to Enhance Hand Gestures for Future AR Smartglasses with Wristband That Can Read Nerve Signals

While Apple has been rather secretive about its work on AR wearables (despite the abundance of leaks revealing details about their unannounced devices), Facebook has been rather chatty about its plans for smartglasses.

The company's latest missive on its AR wearables development focuses not on the smartglasses but on a peripheral that would improve gesture recognition for virtual interfaces.

• Don't Miss: Why Facebook Co-Founder's Call for a Breakup Is Focused on the Wrong Target — AR Is Zuckerberg's Next Big Data Bonanza

On Thursday, Facebook Reality Labs published a preview of the progress it has made with human-computer interfaces that utilize artificial intelligence to read user intent by reading brain signals, technology that founder and CEO Mark Zuckerberg has talked about at length in the past.

The company admits that the ultimate form of this technology is years away, but smart wristband technology is a realistic short-term approach that it could apply to its forthcoming AR smartglasses.

Developed by CTRL-Labs (which Facebook acquired in 2019), the wristband uses electromyography (EMG) sensors to read nerve signals and interpret commands for the movement of hands and fingers.

While hand tracking without peripherals is already possible through AR and VR headsets, evidenced by HoloLens 2, Magic Leap, and Facebook's own Oculus Quest 2, EMG would improve the system by reading the intent of user actions. So, instead of tracking hand movements with camera-based sensors, EMG enables Facebook to essentially "wiretap" your central nervous system.

"What we're trying to do with neural interfaces is to let you control the machine directly, using the output of the peripheral nervous system — specifically the nerves outside the brain that animate your hand and finger muscles," says FRL Director of Thomas Reardon, director of neuromotor interfaces for Facebook Reality Labs who arrived at the company via the CTRL-Labs acquisition, in the blog post.

Facebook outlines several potential AR use cases that the EMG wristband can enable or improve upon. The initial capabilities of the wristband will be limited to basic pinch and release gestures for executing commands, such as confirming predictive tasks delivered by virtual assistants over in-ear monitors that Facebook Reality Labs previewed in a previous blog post.

Eventually, the system will expand into being able to click and drag virtual objects in space. In addition, it will enable faster typing on virtual keyboards, with machine learning that learns your typing style and predictive interfaces that can infer user intent based on context, such as going outside for a jog.

The wearable will also support haptic feedback to add tangible sensations to commands. For example, the distinctive action of shooting a bow and arrow would be accompanied by haptic feedback simulating the resistance of the bowstring as it is pulled back.

The company is testing two prototypes with different haptic feedback mechanisms. One is called Bellowband, which includes eight pneumatic bellows that deliver pressure and vibration patterns. The other is dubbed Tasbi (short for Tactile and Squeeze Bracelet Interface) and employs six vibrotactile actuators and a novel wrist squeeze mechanism.

Facebook Reality Labs also addresses privacy concerns, because, well, its parent company has a certain...reputation. To ensure ethical progress, the Facebook team will rely on publishing research in peer-reviewed journals, collaborating with the academic community to address industry issues, employing its own ethicists to work with the technical teams, and being transparent about what they are working on (hence the behind-the-scenes reveal of its HCI work).

The company's eagerness to develop AR smartglasses has become intertwined in its public spat with Apple. Zuckerberg recently showed his hand, viewing AR and VR as a means to circumventing the restrictions placed on its mobile apps by Apple. That refrain is implied in more utopian language by his executive in charge of AR/VR, where AR glasses provide virtual freedom.

"Imagine being able to teleport anywhere in the world to have shared experiences with the people who matter most in your life — no matter where they happen to be," said Andrew Bosworth, vice president of augmented and virtual reality at Facebook and head of Facebook Reality Labs. "That's the promise of AR glasses. It's a fusion of the real world and the virtual world in a way that fundamentally enhances daily life for the better."