Opinion: Virtual Reality is enjoying a revival of late thanks to advances in hardware and software

By Ann McNamara, Texas A+M University

Virtual Reality has the potential to transform the way we work, rest and play. We are seeing use cases as diverse as education and pain management, with new applications being imagined every day. VR technology comes with new challenges, and many obstacles need to be overcome to ensure a good user experience. One of those challenges facing VR developers is how to best deliver information to the user.

Once a user is placed in VR, usually by wearing a Head Mounted Display (HMD), the creator of the virtual world has no control over how the user will interact with that world. Users can move through the virtual scene however they wish. VR is fully immersive and allows them to look at whatever interests them in the Virtual Environment (VE) and to move through the environment without a particular script or path. 

From RTÉ News, a report on VR trials at St James's Hospital in Dublin

Contrast this to a movie. Directors and cinematographers have full control over what the audience sees and when they see it. There is a predefined sequence of shots each with its own story point to convey. Directors work really hard when crafting a movie to correctly frame the story in an optimal way to ensure that the audience doesn't miss a story beat. There is no danger of being confused or disoriented. They use the camera, and other cinematographic "tricks" to draw viewer attention. In this manner, the audience is guided to when and where on the screen they should focus to best understand the story. 

The opposite is true in VR. The immersive nature allows the user to effectively be the camera; no such tricks are commonly available to communicate the story points or ensure the user interacts with the VE in the manner the world builder intended. In extreme cases, this freedom can lead to confusion in VR, which in turn may lead the user to miss the story or key elements. When this happens users become confused or disoriented, or even physically sick.

The opportunity to capitalise on VR to influence gaze and visually guide the viewer through a scene has gone largely unexplored

Research at Texas A&M University is looking at ways to mitigate this confusion by investigating new ways to guide user attention in immersive three-dimensional (3D) VEs.  Fortunately, we can implement subtle tricks to manipulate the VEs to ensure that users are guided to critical information. 

The opportunity to capitalise on VR to influence gaze and visually guide the viewer through a scene has gone largely unexplored. We are working to develop a system, based on eye movements that can influence where viewers look in a scene, both spatially and temporally. This work builds on known heuristics of image saliency and visual perception to incorporate innovative ways to attract and focus attention on visual information in VR applications.

Our work takes advantage of the dynamic nature of VEs to virtually enhance physical elements, such as paintings, by manipulating image characteristics such as contrast and focus in an effort to attract user attention. We can highlight information in the VE to give visual distinction to POIs or points of interest in a visually rich image such as a painting. 

If we need the user to focus on another area of the virtual environment, we can use subtle techniques to attract attention to these regions

We are also looking at more subtle approaches to guide a user through a VE by presenting cues to the peripheral vision where we, as humans, are more attuned to the motion. Once the user detects motion in their peripheral vision, they turn to look at what is causing the motion. As the user re-directs their gaze, we can turn off the motion cue before they actually see it. It is kind of like when you think you see something from the corner of your eye, then when you look there was nothing there. By repeating this pattern, we can guide users to look at the most pertinent region or object in a VE without degenerating the visual experience of the user. 

We are looking at techniques that succeed at not only drawing the eye but also boost learning and recall. Our experiments are designed to help us determine the most effective solution for influencing gaze, and will then drill deeper to investigate how gaze direction impacts recall and understanding.

In summary, we use eye tracking to alleviate usability issues surrounding information presentation in immersive VEs. The integration of eye tracking allows us to determine where the viewer is focusing their attention. If we, as the content creators and world builders, need the user to focus on another area of the VE we can use subtle techniques to attract attention to these regions and also we can confirm we are doing so successful as we continually track the users gaze.  Advancing these approaches we hope to make the VR experience more comfortable, safe and effective for the user. 

The author will be speaking at ARVR Innovate which takes place on Thursday May 10 in Dublin's RDS. 

The views expressed here are those of the author and do not represent or reflect the views of RTÉ