What you need to know
- Multiple rumors indicate Apple is working on a VR headset.
- A new patent reveals an alert system for hailing users who are immersed in virtual reality.
- A future Apple headset could recognize when someone is trying to get your attention in VR, and even who that person is.
A new patent from Apple has revealed how a future Apple VR headset could one day detect when someone is trying to get your attention whilst you're immersed in virtual reality.
The patent is titled 'System and Method For User Alerts During An Immersive Computer-Generated Reality Experience' and covers systems and methods for hailing the user of a virtual reality headset. The patent explains:
A head-mounted display may be used to present a computer-generated reality application (e.g., including audio and video signals) to a user wearing the head-mounted display. While using a computer-generated reality application, a user's ability to sense and be aware of their physical surroundings can be substantially impaired. Using a computer-generated reality application could make it difficult for other people near the user to get the attention of the user. Systems and processes for facilitating the hailing of user immersed in computer-generated reality application are described herein.
The system features a head-mounted display, as well as an image sensor and a microphone that can detect when the user of a VR/AR headset is being "hailed":
The systems may include image sensors and/or microphones attached to the head-mounted display or positioned nearby so as to capture images and/or sound from a vicinity around the user wearing the head-mounted display. For example, data from these sensors may be analyzed to detect a person and/or indications that the person is trying to hail the user, e.g., to attract the attention of the user. For example, if the person speaks to the user, audio triggers (e.g., key phrases such as "hello," "excuse me," or a name of the user) can be recognized in the speech signal. In some implementations, a direction of arrival for a speech signal may be determined and compared to the view angle for the person to verify that the speech signal is coming from the nearby person. In some implementations, a face of the person (e.g., including one or more facial landmarks, such as a mouth, a nose, or pupils) may be detected recognized as matching a registered face of a known person (e.g., a friend or colleague of the user). In some implementations, the face can be detected and an orientation of the face with respect to the head-mounted display can be determined to assess whether the person is facing toward the user. In some implementations, eyes of the person can be analyzed more closely to determine a gaze direction, in order to assess whether the person is looking at the user.
Not only can the system alert you to someone trying to get your attention in VR, it can also use image detection to figure out who exactly might be trying to talk to you. When a hail event is detected, the headset could display an audio and visual alert.
Multiple reports indicate Apple is working on a mixed-reality headset that could be used for gaming and communication.
