Here's how Apple Vision Pro's most striking feature, called EyeSight, works

Contrary to what it may appear, the front of the Vision Pro headset is not made of transparent glass. Like any other VR headset, it actually has an opaque layer that includes displays, lenses, sensors, and chips, which conceal the wearer's eyes.

The technology behind EyeSight involves an outward display that closely mimics the eyes of the headset user. This display is not only curved but lenticular as well, meaning it shows a slightly shifted image of the eyes depending on the viewing angle. This design choice ensures that the eyes look natural even when viewed from the side, unlike a conventional 2D display.

To generate the separate views required for different angles, EyeSight combines data from two sources. Firstly, it utilizes footage from the four eye-tracking cameras inside the headset. Secondly, it uses a 3D facial scan of the person wearing the headset to create a digital avatar. These data sources are fused together with minimal latency to render the digital image of the eye and its surroundings.

The primary goal of EyeSight is to break the social isolation often associated with VR headsets. Apple aims to enable natural interactions between headset users and people in their immediate environment. The technology allows for eye contact and creates a sense of transparency, making users feel connected and less disconnected from others.

Eyesight automatically switches between a transparent and opaque view depending on the user's activity. When consuming immersive content, it provides an opaque view, but when interacting with people or the environment, it switches to a transparent view, allowing others to see the user's eyes.

Although the press has not yet had the opportunity to try EyeSight during initial hands-on experiences with Apple Vision Pro, Apple is likely still refining the feature. It's worth noting that some experts anticipate limitations, such as a potentially dim and low-resolution image, as well as restrictions on viewing angles due to the lenticular technology used.