An insider’s guide to designing near-eye displays
There are a variety of virtual reality (VR) and augmented reality (AR) near-eye display (NED) solutions currently in development, and the viability of a visual experience that seamlessly blends digital content with the physical world continues to grow. Let’s take an in-depth look at some of the top challenges for designing a compelling see-through near eye display solution that “seamlessly” blends the digital world with the physical world.
Figure 1: Near-eye displays are still in their infancy but careful attention to the details of a design can help it deliver vivid, clearly-defined images with precise orientation within a synthetic visual field.
There are many situations where the technical refinement of a NED solution is not just nice-to-have, but important to the usability of the NED. Imagine a surgeon or EMT is wearing a NED solution as a supplemental tool during a medical procedure. In such an environment, a clean, unobtrusive experience is critical. Or consider a video game player, for whom a very low display lag is required in order to deliver a seamless, real-time experience.
In either case, a compelling visual experience depends on minimizing latency (delay) of the image being displayed, maximizing the optical contrast, and maximizing the field of view (FOV) of the information being displayed.
Display Latency – The key to creating a real-time experience
First considering system latency, there are many system-level components which contribute to the latency experienced by the user. For our purposes, we will focus on the portion associated with the display engine, which can be divided into two components:
Display (Pixel) Latency = Pixel Data Update Time + Pixel Switching Time
The first component, called “pixel data update time,” is the time it takes for the display device to “load” a new data value into a display pixel. For many display engine architectures, this is one or more frame times, when measured from the input to the engine. Assuming a one frame delay, this is about 16.67ms for a 60Hz source, which is common because many modern display technologies include a frame memory for facilitating image processing. For some display engines, pixel data update time can be two or more frames.
The second component of the display latency is “pixel switching time,” which is the time it takes for a pixel to switch from its current state (on or off) to the opposite state. The end of the pixel switching time is when the pixel has settled enough that the human observer can clearly perceive the new data.
The pixel data update time plus the pixel switching time sets the total display lag time as perceived by a human observer. A display latency time of 16.67ms is often considered very good, but some displays can have lag times of 60ms or more.
Texas Instruments DLP® Pico chips have some of the fastest pixel speeds available and can flip each digital micromirror (pixel) thousands of times per second, thereby reducing the display latency, and thus supporting display frame rates up to 120Hz while maintaining high image quality.
Contrast – The key to visually blending digital content with the real world
In addition to delivering a low-latency, real-time experience, the ideal NED solution should deliver transparent content with high clarity so as to not obstruct the end user’s view of the real world. For example, if the data to be displayed is only using 20 percent of the display-device pixel array, the other 80 percent should be practically invisible to the user, thereby blending the digital content with the real world.
It is important to note that within a see-through NED optical system, the image is not being displayed on a semi-transparent surface (i.e., eyeglass lenses). Displaying on a semi-transparent surface would not be effective since such a surface would, by definition, be very close to the user’s eye, and the eye cannot comfortably focus on something so close. Rather than creating an image on a surface, the optical system forms an optical pupil and the human eye acts as the last element in the optical chain – thereby creating the final image on the eye’s retina.
Figure 2: – Block diagram of a DLP-based NED.
A common see-through NED optical system will include a waveguide optical element which collects the light at the input and relays it towards the user’s eye. Such an arrangement not only forms the necessary optical pupil, but it also allows the micro-display, optics and illumination to be positioned so as to not obstruct the user’s view.
Now that we understand the optical system, how do we ensure that the transparent areas of the displayed image do not obstruct the user’s view? The best way to achieve this is to maximize the optical systems contrast ratio. The image below illustrates the impact that contrast can have on the display as seen by the NED user.
Figure 3:This split image illustrates the profound impact contrast ratio can have on an image’s quality and readability. (Note: Photo is a simulation only and not from an actual near-eye display.)
Numerous elements in the NED design can impact the contrast ratio. The principle contributors include the F-number of optical design, and the availability of advanced image processing algorithms. With some micro-display devices, fill-factor can also affect contrast, but this is typically to a lesser degree.
The F-number of the optical design indicates the ratio of the lens’ focal length to the diameter of the entrance pupil. A higher F-number enables a higher contrast ratio – as well as reduced optical complexity and smaller optics size. Although a high F-number will give higher contrast, it must also be balanced with the required field of view – because a higher F-number not only increases contrast, but simultaneously decreases the field of view.
Advanced image processing can also improve the contrast ratio by intelligently managing the RGB illumination (i.e., LED brightness) in conjunction with digital gain applied to each image frame. For example, the new TRP chipsets from TI DLP Products feature the IntelliBright™ suite of algorithms which includes a function called Content Adaptive Illumination Control (CAIC). This algorithm can intelligently adjust the image brightness depending on image content and ambient lighting conditions. This not only results in optimal image brightness and image contrast, it also optimizes system power consumption; another important factor in NEDs.
Figure 4: A typical DLP element
Keeping the see-through experience natural with a wider field of view
The human eye has an almost 180-degree horizontal FOV. Augmented reality headsets typically have a 20 to 60 degree FOV, which is sufficient to result in a natural viewing experience. In comparison, typical smart glasses solutions tend to have a smaller FOV that the user must unnaturally glance at periodically. The trend in most see-through NED applications is toward a wider FOV. A wider FOV will also allow the display to overlay more content across the user’s natural view of the real world, thereby providing a higher quality viewing experience.
Field of view is controlled by three key design factors: micro-display array diagonal size, F-number of the optics, and pupil size at the end of the waveguide. Several trade-offs across these factors should be considered: A larger array diagonal size will provide a higher field of view and also higher resolution in most cases, however this will also grow the system size since the diagonal size typically translates to larger optics. A lower F-number optical design will lead to a larger FOV; however this will also increase optics size and decrease contrast. As the pupil size increases, the FOV decreases. For example, a 5mm pupil size can achieve a 45-degree FOV while a 10mm pupil size at the same F-number will achieve less than 25 degree FOV.
With many see-through NED solutions in development, delivering visual experiences that seamlessly blend digital content with the physical world is critical. The design challenges require trade-offs that directly impact the end-user experience. For more information about some of these tradeoffs, download the DLP Technology for Near Eye Display White Paper and visit TI’s E2E community to explore design solutions with TI experts.
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
