How to use Screen Recognition on your iPhone

Apple is one of those brands that build features for users with disabilities. VoiceOver is a prime example of its efforts to make the iPhone easy to use for the visually impaired. The Cupertino giant is going beyond manual labour with its new screen recognition feature on iPhone. With screen recognition, anyone can have their iPhone's screen read to them, even if the apps are not optimized for VoiceOver. Here's how to activate and use it.

What is Screen Recognition

Screen Recognition enhances the already-existing VoiceOver feature. It detects the layout of elements in the user interface, especially in apps that have not been optimized for VoiceOver.

Up until now, every element of an app's interface had to be manually labeled to use VoiceOver for the visually impaired. Now, Apple uses machine learning to automatically identify and label every button, slider, and tab.

Screen Recognition on iPhone is essentially a computer vision system. It has been trained on thousands of images of apps in use. The algorithms have learned what a button looks like, what icons mean, and how various on-screen elements differ from each other.

It works at pixel-level to label every item on the screen when users invoke the feature. It can recognize every element from images to common buttons and icons like home or back, and labels '...' menus to understand what they mean.

How is it useful, anyway? Screen recognition's ability to determine UI elements — from the simple to the most complicated ones — even if developers did not label them, fixes a major problem. The changing standards and updates can make things not as accessible. VoiceOver's screen recognition feature automatically detects the updated controls and helps visually impaired people understand every minor possible UI element.

Imagine you updated an app and it received a design overhaul. This update would confuse the visually disabled if new labels are missed. But screen recognition benefits the users with its ability to automatically detect UI elements, making sure every button, tab, and menu is labeled.

How to turn on Screen Recognition

1. Launch the Settings app from your Home screen.
2. Tap on Accessibility.

3. Tap on VoiceOver under the Vision category.

   Source: Prakhar/iMore

4. Tap on VoiceOver Recognition.

5. Tap on Screen Recognition.

   Source: Prakhar/iMore

6. Toggle it on.
7. Go back.

8. Toggle VoiceOver on.

   Source: Prakhar/iMore

Note: You need to have Wi-Fi or LTE switched on as Screen Recognition requires downloading some data to work.

How to add Screen Recognition to Rotor

You might want to add Screen Recognition to the Rotor to enable the accessibility feature while using VoiceOver manually. To add Screen Recognition to Rotor:

1. Go to Settings.
2. Tap on Accessibility.

3. Tap on VoiceOver under the Vision category.

   Source: Prakhar/iMore

4. Tap on Rotor

5. Scroll down a bit and tap on Screen Recognition.

   Source: Prakhar/iMore

Any questions?

Let us know in the comments below if you have any questions.