What you need to know
- Apple has told developers about its new framework, Vision.
- It can be used to help an app detect body and hand poses in photos and videos.
- It can be used to analyze the movement, and even trigger an action based on a gesture, like taking a photo.
A new Apple developer session for WWDC 2020 has detailed how developers can use Apple's new 'Vision' framework to detect body and hand poses in photos and videos.
Apple is adding a new capability for developers to iOS 14 and macOS Big Sur, which will allow developers to make apps that can detect hand and body poses and movements in videos and photos. From the video summary:
The technology can be used for apps in fields like sports analysis, classification apps and image analysis, object tracking, and more. Not only can it be used to analyze what's happening on the screen, but it can also be used to detect gestures that then trigger action responses. For example, making a certain pose to a camera with your hand could make your iPhone automatically take a photo. The framework has a few limitations, it doesn't perform as well with people wearing gloves, bending over, being upside down or wearing flowy or robe-like clothing.
Master your iPhone in minutes
iMore offers spot-on advice and guidance from our team of experts, with decades of Apple device experience to lean on. Learn more with iMore!
Stephen Warwick has written about Apple for five years at iMore and previously elsewhere. He covers all of iMore's latest breaking news regarding all of Apple's products and services, both hardware and software. Stephen has interviewed industry experts in a range of fields including finance, litigation, security, and more. He also specializes in curating and reviewing audio hardware and has experience beyond journalism in sound engineering, production, and design.
Before becoming a writer Stephen studied Ancient History at University and also worked at Apple for more than two years. Stephen is also a host on the iMore show, a weekly podcast recorded live that discusses the latest in breaking Apple news, as well as featuring fun trivia about all things Apple. Follow him on Twitter @stephenwarwick9