Source: Prakhar Khanna/iMore
Apple is one of those brands that build features for users with disabilities. VoiceOver is a prime example of its efforts to make the iPhone easy to use for the visually impaired. The Cupertino giant is going beyond manual labour with its new screen recognition feature on iPhone. With screen recognition, anyone can have their iPhone's screen read to them, even if the apps are not optimized for VoiceOver. Here's how to activate and use it.
What is Screen Recognition
Screen Recognition enhances the already-existing VoiceOver feature. It detects the layout of elements in the user interface, especially in apps that have not been optimized for VoiceOver.
Up until now, every element of an app's interface had to be manually labeled to use VoiceOver for the visually impaired. Now, Apple uses machine learning to automatically identify and label every button, slider, and tab.
Screen Recognition on iPhone is essentially a computer vision system. It has been trained on thousands of images of apps in use. The algorithms have learned what a button looks like, what icons mean, and how various on-screen elements differ from each other.
Source: Prakhar Khanna/iMore
It works at pixel-level to label every item on the screen when users invoke the feature. It can recognize every element from images to common buttons and icons like home or back, and labels '...' menus to understand what they mean.
How is it useful, anyway? Screen recognition's ability to determine UI elements — from the simple to the most complicated ones — even if developers did not label them, fixes a major problem. The changing standards and updates can make things not as accessible. VoiceOver's screen recognition feature automatically detects the updated controls and helps visually impaired people understand every minor possible UI element.
Imagine you updated an app and it received a design overhaul. This update would confuse the visually disabled if new labels are missed. But screen recognition benefits the users with its ability to automatically detect UI elements, making sure every button, tab, and menu is labeled.
How to turn on Screen Recognition
- Launch the Settings app from your Home screen.
- Tap on Accessibility.
Tap on VoiceOver under the Vision category.
Source: Prakhar/iMore
- Tap on VoiceOver Recognition.
Tap on Screen Recognition.
Source: Prakhar/iMore
- Toggle it on.
- Go back.
Toggle VoiceOver on.
Source: Prakhar/iMore
Note: You need to have Wi-Fi or LTE switched on as Screen Recognition requires downloading some data to work.
How to add Screen Recognition to Rotor
You might want to add Screen Recognition to the Rotor to enable the accessibility feature while using VoiceOver manually. To add Screen Recognition to Rotor:
- Go to Settings.
- Tap on Accessibility.
Tap on VoiceOver under the Vision category.
Source: Prakhar/iMore
- Tap on Rotor
Scroll down a bit and tap on Screen Recognition.
Source: Prakhar/iMore
Any questions?
Let us know in the comments below if you have any questions.
We may earn a commission for purchases using our links. Learn more.

Apple has released macOS Monterey 12.5 beta for public beta testers
Are you ready to take the next version of macOS out for a spin? Here's how to install the public beta of macOS Monterey on your computer.

Review: Is the Bowers & Wilkins Panorama 3 soundbar the one for you?
If you're in the market for a new soundbar, it's worth considering Bowers & Wilkins' latest entrant, the Panorama 3.

Watch the stunning second trailer for Apple TV+ series 'Prehistoric Planet'
Apple TV+ has today shared a second trailer for the upcoming dinosaur series 'Prehistoric Planet,' a five-night event that will feature Sir David Attenborough.

Make the most of MagSafe features with these cool accessories
The iPhone 12 is the first iPhone with built-in MagSafe magnets. Make the most of this awesome feature with the best MagSafe accessories to improve convenience and functionality.