Iphone Screen RecognitionSource: Prakhar Khanna/iMore

Apple is one of those brands that build features for users with disabilities. VoiceOver is a prime example of its efforts to make the iPhone easy to use for the visually impaired. The Cupertino giant is going beyond manual labour with its new screen recognition feature on iPhone. With screen recognition, anyone can have their iPhone's screen read to them, even if the apps are not optimized for VoiceOver. Here's how to activate and use it.

Screen Recognition on iPhone

What is Screen Recognition

Screen Recognition enhances the already-existing VoiceOver feature. It detects the layout of elements in the user interface, especially in apps that have not been optimized for VoiceOver.

Up until now, every element of an app's interface had to be manually labeled to use VoiceOver for the visually impaired. Now, Apple uses machine learning to automatically identify and label every button, slider, and tab.

VPN Deals: Lifetime license for $16, monthly plans at $1 & more

Screen Recognition on iPhone is essentially a computer vision system. It has been trained on thousands of images of apps in use. The algorithms have learned what a button looks like, what icons mean, and how various on-screen elements differ from each other.

Screen-recognition-on-iOSSource: Prakhar Khanna/iMore

It works at pixel-level to label every item on the screen when users invoke the feature. It can recognize every element from images to common buttons and icons like home or back, and labels '...' menus to understand what they mean.

How is it useful, anyway? Screen recognition's ability to determine UI elements — from the simple to the most complicated ones — even if developers did not label them, fixes a major problem. The changing standards and updates can make things not as accessible. VoiceOver's screen recognition feature automatically detects the updated controls and helps visually impaired people understand every minor possible UI element.

Imagine you updated an app and it received a design overhaul. This update would confuse the visually disabled if new labels are missed. But screen recognition benefits the users with its ability to automatically detect UI elements, making sure every button, tab, and menu is labeled.

How to turn on Screen Recognition

  1. Launch the Settings app from your Home screen.
  2. Tap on Accessibility.
  3. Tap on VoiceOver under the Vision category.

    Tap on Settings, Go to Accessibility, and tap on VoiceOverSource: Prakhar/iMore

  4. Tap on VoiceOver Recognition.
  5. Tap on Screen Recognition.

    Tap on Voice Recognitions and go to Screen RecognitionSource: Prakhar/iMore

  6. Toggle it on.
  7. Go back.
  8. Toggle VoiceOver on.

    Toggle Screen Recognition on. Go back and Toggle Voiceover onSource: Prakhar/iMore

Note: You need to have Wi-Fi or LTE switched on as Screen Recognition requires downloading some data to work.

How to add Screen Recognition to Rotor

You might want to add Screen Recognition to the Rotor to enable the accessibility feature while using VoiceOver manually. To add Screen Recognition to Rotor:

  1. Go to Settings.
  2. Tap on Accessibility.
  3. Tap on VoiceOver under the Vision category.

    Tap on Settings, Go to Accessibility, and tap on VoiceOverSource: Prakhar/iMore

  4. Tap on Rotor
  5. Scroll down a bit and tap on Screen Recognition.

    Go to Rotor and tap on Screen RecognitionSource: Prakhar/iMore

Any questions?

Let us know in the comments below if you have any questions.

Get More iPhone

Apple iPhone

We may earn a commission for purchases using our links. Learn more.