Technology

Eye-sensing Light Field Display

Delivering 3D Creators' Visions to Customers the Way They Intended

3D displays have great potential for various applications, such as showroom displays and entertainment. Conventional 3D displays, however, suffer from issues such as low content resolution, the hassle of wearing 3D glasses, and 3D motion sickness. The Eye-sensing Light Field Display (ELFD) developed by Sony is a new type of 3D display that has the potential to solve these issues. By working together with creators to develop not only the content but also the production environment, we aim to popularize this system in a wide range of fields.

Delivering a "Spatial-Reality Experience" with Sony's Unique Technologies

What kind of display is an ELFD?

Ban Kawamura: ELFD is a display that allows you to see objects in 3D volumetrically, as if they were right in front of you. The realism remains the same even from different angles. It allows you to experience high-definition 3DCG (3D Computer Graphics) images with the naked eye, and we expect this technology to be applied in a wide range of applications such as monitors for 3DCG creators, showroom displays, and entertainment. For example, before creating a mockup of a small product like a remote controller, you could check the volume, texture, and appearance with ELFD.

A 15-inch diagonal prototype was exhibited at CES2020. As long as the content fits within these dimensions, it can be displayed as a high-definition, three-dimensional object.

What kind of technologies are used?

Kazuki Yokoyama: ELFD uses three of Sony's proprietary technologies: (1) High-speed, high-accuracy, real-time sensing technology, (2) Real-time light field rendering technology, and (3) High-precision 3D display technology.

1. High-speed, high-accuracy, real-time sensing technology: In order to make the viewer feel as if the object is real, it is necessary to continuously display the correct image and viewpoint based on the position of the user's eyes. ELFD uses Sony's unique high-speed vision sensors and face recognition algorithms to achieve accurate eye detection with minimal delay. The positions of both eyes are calculated in real time, not only for the horizontal and vertical planes but also for depth.
2. Real-time light field rendering technology: Based on the user's position, ELFD calculates how the object would look through the user's eyes if it were placed inside the display and converts this information into a sort of deceptive picture on the display's surface.
3. High-precision 3D display technology: Although there have been other 3D displays that do not require glasses, they could not accurately present 3D images due to low resolutions and crosstalk between the eyes. The ELFD display makes full use of Sony's high-speed vision sensor with a unique micro-optical lens that utilizes eye tracking, significantly reducing crosstalk compared to conventional 3D displays.

Please tell us about Sony's previous 3D display development prior to the development of ELFD.

Yokoyama: There was a time about ten years ago when 3D was all the rage. At that time, Sony was developing a 3D TV with 3D glasses. However, while the 3D TVs initially attracted a lot of interest, they did not take hold in the market because some people found it difficult to see sterically, and some even felt sick while watching them.

A few years later, we began considering a new type of 3D display that incorporated binocular parallax adjustments. By applying this display technology and examining the optimal placement of the display panel and current viewpoint image, we were able to develop the prototype of ELFD.

Faithfully Reproducing the "Creator's Intent"

What was important to you during the development?

Takaomi Kimura: The main concept behind the development of ELFD was facilitating and helping to bring the "Creator's Intent" to life. This technology is not only impressive on a technical level, but it's also a vehicle through which creators can express their ideas. Therefore, during the development process, great emphasis was placed on dialogue with creators. We wanted to allow creators to produce ELFD content with tools they are familiar with, so we created a software development kit (SDK) that supports Unity and Unreal Engine.

What was the reaction of people who experienced ELFD?

Thaisa Yamamura: At CES 2020, we had a great turnout, with people queuing up to try ELFD. We received a lot of positive feedback, as well as excellent suggestions that we had not thought of.

Exploring New Possibilities for 3D Content with Creators

What do you want to achieve in the future?

Kimura: XR (AR/VR) has great growth potential in entertainment applications and many B2B fields. Yokoyama: We would like creators in a wide variety of fields to produce high-quality content so that we can spread this technology and make it a permanent part of people's daily lives. Kawamura: In the future, we are looking to apply it to the education sector and VR display devices as well as expanding it into fields like B2B solutions and content distribution.