What Is EyeSight on Apple’s Vision Pro Headset? How Does It Work?

0

Apple’s $3,499 Vision Pro headset has a game-changing feature called EyeSight. Built into visionOS, EyeSight reduces the feeling of isolation when wearing the headset.




Vision Pro makes you feel like people nearby are part of your augmented reality (AR) experience. It also indicates to others around you whether you can see them or are immersed in AR by providing visual cues about what you’re focused on.

Follow along to learn everything about EyeSight, including how it works, the different immersion modes it supports, and more.


What Is the Vision Pro’s EyeSight Feature?

Vision Pro digital eyes using EyeSight
Image Credit: Apple

An interplay of hardware and software, EyeSight is one of the best features of the Vision Pro headset. Its primary purpose is to show others a digital version of your blinking eyes.


“Your eyes are a critical indicator of connection and emotion, so Vision Pro displays your eyes when someone is nearby,” Apple said during the WWDC 2023 keynote. The effect makes the device appear transparent. In addition, EyeSight can render various motion backgrounds to inform people in the real world what you’re currently focused on.

In an Apple Newsroom post, Apple calls the feature “an extraordinary innovation that helps users stay connected with those around them.” To be sure, there’s not quite anything like EyeSight on other AR/VR headsets.

How Does EyeSight Work on the Vision Pro?

First-person view of talking to a friend while in augmented reality wearing Apple Vision Pro headset
Image Credit: Apple

An outward-facing OLED panel on the headset’s front uses a lenticular lens that Apple says projects “the correct perspective of your eyes to each person looking at you.”


When the Vision Pro detects you’re consuming media or using an app, it’ll render appropriate graphics on the EyeSight display to provide visual cues. However, when someone approaches you, the Vision Pro will show your eyes instead.

To show your eyes realistically, EyeSight uses data from internal infrared cameras, two per eye, to capture your eye movement and facial expressions. This data is fed to a machine learning algorithm which combines it with your digital Persona, created when setting up the device using the front-facing TrueDepth camera.

What others see is a natural representation of the area around your eyes rather than your actual eyes.


The Different EyeSight Modes on the Vision Pro

EyeSight supports several visual cues to let others know what you’re doing in AR, like fully immersed in AR, taking pictures, etc. EyeSight automatically switches modes depending on what you’re doing at any given moment. As such, the EyeSight feature is one of the coolest applications of AR technology in everyday life we’ve seen to date.

Transparent Mode Displays Your Digital Eyes

person conversating while wearing an Apple Vision Pro headset
Image Credit: Apple

This mode lets you see the other person, revealing your eyes to them. When the headset’s sensors pick up a person approaching you, EyeSight will temporarily make them “break” into your AR/VR world so you know they’re around.

At the same time, the feature will display your eyes to the person in front of you so they know they have your attention. When there’s no one in the room, EyeSight will shut off and won’t display your eyes or any custom background.


Full Immersion Mode Hides Your Eyes

Woman using augmented reality apps in Apple Vision Pro headset
Image Credit: Apple

When fully engulfed in AR or VR, EyeSight animates a colorful pattern on the outward-facing display. The effect is somewhat similar to the Siri orb animation, and those around you will know you’re busy with activity. EyeSight may also render your eyes semi-transparently when using an app, depending on the level of immersion.

Capture Mode Replaces the Shutter Sound

person capturing videos on Vision Pro
Image Credit: Apple


EyeSight enters this mode when taking spatial photos and videos using the headset. A visual indicator for the media capture mode is a white, fog-like pattern that animates as you record or flashes when you take a picture.

The visual indicator serves as an essential privacy indicator to folks around you that you’re recording them, similar to the camera shutter sound on your iPhone.

EyeSight Makes Your AR/VR Experience Less Isolating

What sounded like a crazy rumor has turned out to be true. As a feature unique to the Vision Pro, EyeSight is one of Apple’s biggest advantages over competing AR/VR headsets. If the whole point is to make wearing a face computer a bit more human, EyeSight delivers.

That said, whether we’ll get used to people wearing Vision Pro headsets—with their digital eyes bizarrely peering out of the device’s front—in a public setting remains to be seen.

link

Leave a Reply

Your email address will not be published. Required fields are marked *