What you need to know
-Apple is experimenting with ideas around how to best control augmented and virtual reality experiences. - Device patent points to wearable finger controllers. - Smartly, the design leaves fingertips free to use touchscreens.
Apple has long been rumored to be making a virtual reality / augmented reality device, and though its summer software showcase failed to reveal the goods last week, a newly uncovered patent shows Apple's continued exploration of the mixed reality space.
The Apple VR/AR headset may not have been revealed at WWDC 2022, but the patent (number 11,360,558, filed with the United States Patent Office) reveals a set of devices worn like thimbles around a user's fingers, designed to control virtual and mixed reality experiences, and interact with objects within them.
The patent describes arrays of one dimensional and two dimensional sensor elements, taking advantage of capacitive touch sensors, cameras and sensors tracking optical and ultrasonic data. Force and inertial movement can also be tracked. Essentially, you've got something that will let a wearer control elements in 3D space – the VR or AR equivalent of a pointer or mouse, similar to VR controllers we see with Oculus / Meta Quest devices at present, but reduced to their essential elements so as to fit on a user's finger, letting gestures, movements, and inputs be tracked through the air.
The patent specifically references viewing this content through a 'head-mounted device', with haptic feedback included to help a wearer understand a sense of presence in the digital world and through its interactions. It also describes how a connected AR system could better recognise real-world objects through their interaction and proximity to the finger-worn devices, and as such overlay differing information onto physical objects through a headset or similar display based on their rotation and angle. Interestingly, the patent leaves the wearer's fingertip exposed – presumably to keep them free for interacting with a touchscreen device like its best iPhones, or a MacBook's keyboard, anchoring a wearer to their real-world devices.
Pointing a finger into the future
Apple is on the cusp of entering a busy AR/VR space, with lots of competition from the likes of Meta, Sony PlayStation, Snap and other companies betting bigger on the coming importance of the metaverse. Innovation will be key to standing out, so it's good to see Apple experimenting with fresh ideas.
It's interesting to see Apple considering finger based devices too, given that Meta, for instance, is working hard to make hands-free hand and finger tracking the eventual norm, using a headset's onboard camera and space sensors to identify digits and hands and have them tracked accordingly.
But it's a sometimes hit-and-miss affair, and no system has yet managed to offer hands-free control with the same level of accuracy that a held device, be that wand or controller, can offer. Apple's patent here seems to be a concessionary step between both solutions – small enough to be unobtrusive to an immersive mixed reality experience, yet with enough sensors and tech to augment what a person's fingers can achieve.
As ever, a patent is no guarantee of a device ever making it out of the idea phase – it's merely to protect Apple's intellectual property. But it's an encouraging sign as to what Apple may well be cooking up for its augmented and virtual futures.
Master your iPhone in minutes
iMore offers spot-on advice and guidance from our team of experts, with decades of Apple device experience to lean on. Learn more with iMore!
Gerald Lynch is the Editor-in-Chief of iMore, keeping careful watch over the site's editorial output and commercial campaigns, ensuring iMore delivers the in-depth, accurate and timely Apple content its readership deservedly expects. You'll never see him without his iPad Pro, and he loves gaming sessions with his buddies via Apple Arcade on his iPhone 14 Pro, but don't expect him to play with you at home unless your Apple TV is hooked up to a 4K HDR screen and a 7.1 surround system.
Living in London in the UK, Gerald was previously Editor of Gizmodo UK, and Executive Editor of TechRadar, and has covered international trade shows including Apple's WWDC, MWC, CES and IFA. If it has an acronym and an app, he's probably been there, on the front lines reporting on the latest tech innovations. Gerald is also a contributing tech pundit for BBC Radio and has written for various other publications, including T3 magazine, GamesRadar, Space.com, Real Homes, MacFormat, music bible DIY, Tech Digest, TopTenReviews, Mirror.co.uk, Brandish, Kotaku, Shiny Shiny and Lifehacker. Gerald is also the author of 'Get Technology: Upgrade Your Future', published by Aurum Press, and also holds a Guinness world record on Tetris. For real.