Virtual Reality - 9 issues

Introduction

As VR (Virtual Reality) begins its second (or third) cycle of push into the realms of technology, it is worthwhile to check what problems remain to be solved. This article is a review of past research on fundamental issues that have not yet been solved.

Note: In this article, VR will primarily refer to immersiveness in the Visual domain, and specifically with HMDs (Head Mounted Displays).

Current Issues

The performance of HW renderers (GPUs), quality of display devices (specifically AMOLEDs), and low-power processes to achieve longer thermal cycles has increased significantly in the last 2 years. The chief contributor to VR sickness - "latency", is being solved through a combination of Capable GPUs, Sensor fusion, Dedicated software stacks, improved optics, and low-persistence displays.

However, there are other issues that are important to both usability and adaptability to different kind of people, because these have the potential to affect the normal behaviour in the real-world that we still perform majority of our work in. Solving these issues will make VR truly a mass market technology and fulfil its true potential.

# Issue Description
1 Accomodation-convergence conflict In the real world, our eyes work to both focus and converge on a point in space when we look toward it through a natural process called the accommodation-convergence reflex. Because of the flat nature of the screen in front of the eyes in the HMD environment, it causes a conflict in the human visual system. Principles like Monovision are being researched to resolve this [11].
2 Stereopsis inhibition The visual facility that creates depth perception is still evolving in the case of children. Being in a VR environment for a significant period of time can cause lack of depth perception (ie, causes depth inhibition) in children upto 6-7 year old.
3 Strabismus Prolonged exposure to VR environments can accentuate lazy-eye syndrome, or Strabismus, where the brain does not acknowledge (for yet unknown reasons) the availability of input from an eye. While VR can itself be used as a therapy to correct Strabismus to a certain extent, the full impact of VR on this phenomenon is not yet known.
4 Pattern burning Staring at a virtual grid for a significant period of time can give the sensation of burning the pattern into a person's retinal system, quite similar to the other display mechanisms. This effect can get accentuated because of the perceived immersion and/or brightness levels. This can be considered to be the equivalent of continued exposure to loud sounds in the auditory system. Audio devices may be capable of inducing a permanent bilateral sensorineural hearing loss.
5 Past pointing The brain adapts to the constraints of a simulated world. When the simulation is over, for a brief period of time, the brain has to re-adapt. If the human reaches for something he/she may reach past it because VR may affect the perception of objects at a distance. Past-pointing can significantly affect an individual's ability to carry out everyday tasks, per research done by Biocca et al. The fact that the human visual system can adapt to the "new" mechanism of seeing and the associated pseudo-reality that comes along with it, can cause temporary or permanent alteration in hand-eye coordination, for example in AR-VR environments. This is especially true in the case of people operating in a medical/ industrial world, like Doctors performing surgery.
6 Flicker and FoV, Brightness Wider field-of-view displays enhance performance but increase the likelihood of simulator sickness/ flicker. Flicker is more readily perceived in peripheral than foveal vision, so that characteristics such as refresh rate and luminance that give an acceptable display for a narrow field-of-view may cause unacceptable symptoms with a wide field-of-view.

Flicker is related to the luminance, or brightness, of the display, with flicker sensitivity increasing as the brightness increases.

7 Judder The FOV in an HMD is much wider compared to other displays causing judder to be much more evident on an HMD. In addition, virtual images on a monitor appear to be on a surface in the world, in contrast to virtual images on an HMD, which appear to be directly in the world; this causes the perceptual system to have higher expectations for HMD images and to more readily detect deviations from what we’re used to when looking at the real world.
8 Pseudo-corialis effect Head movements made in the presence of only visual cues of self-motion is termed the pseudo-Coriolis effect, and causes motion sickness
9 Other factors Motion sickness is accentuated by the weight of the head-gear. Also, non-meaningful object rendering (ex, wrong wind/shadow direction), decreases the sense of presence.

Conclusion

These are the top issues faced by the VR ecosystem. Other than the lack of content in the VR world, making people "stay" in the VR world for a reasonable period of time is a significant challenge. But being in VR environment for a long period of time does not affect all people the same way. Hidden Path Entertainment founder Jeff Pobst is reported to have been able to use VR headset for almost 15 hrs. Many forums report that people have played in the Rift for upto 30 hours without issues.

It is appropriate to conclude this article with a quote from Biocca (1992). - "Cybersickness is not so much a bug that can be eradicated, but more of a snake in the underbrush that is there to stay, even though its effects may be reduced". Jen-Hsun, CEO of Nvidia, puts it like this in 2016 - " “First of all, VR displays are a little too cumbersome. It has to be much more elegant, being connected by a wire has to be solved. The resolution has to be a lot higher. The physical worlds do not behave according to the laws of physics. The environment you’re in isn’t beautiful enough. We’re going to be solving this problem for the next 20 years.”

Footnote

In the CV1, the lenses use traditional fresnel-style concentric prisms, but with a sloping layer on the rear, which allows “the focus to vary along the vertical axis.”

References

[1] David Hoffman and Ahna/Kurt/Martin in http://jov.arvojournals.org/article.aspx?articleid=2122611

[2][3] http://journal.frontiersin.org/article/10.3389/fnint.2014.00071/full

[4] https://medicine.stonybrookmedicine.edu/surgery/blog/headphones-and-earp...

[5] 1998 issue of PRESENCE (MIT), "Virtual Eyes Can Rearrange Your Body: Adaptation to Visual Displacement in See-Through, Head-Mounted Displays" - http://www.mitpressjournals.org/doi/abs/10.1162/105474698565703#.VwPkll2...

[6] "Side effects of virtual environments : a review of the literature", http://dspace.dsto.defence.gov.au/dspace/bitstream/1947/4079/1/DSTO-TR-1...

[7] http://blogs.valvesoftware.com/abrash/why-virtual-isnt-real-to-your-brai...

[8] Optokinetic Motion Sickness And Pseudo-Coriolis Effects Induced By Moving Visual Stimuli, Dichgans J et al

[9] "Field of View for Virtual Reality Headsets Explained" - http://vr-lens-lab.com/blog/

[10] http://www.digitaltrends.com/virtual-reality/ifixit-teardown-of-oculus-r...

[11] Monovision - http://www.computationalimaging.org/publications/novel-optical-configura...

[12] Stereoscopic 3D - Section on Errors - http://www.sky.com/shop/__PDF/3D/Basic_Principles_of_Stereoscopic_3D_v1.pdf

[13]http://vrguy.blogspot.in/2016/07/key-parameters-for-optical-designs.html - Issues in designing optics for VR

Also refer to Latest news on Graphics at - http://gpupowered.org/node/39

- Prabindh Sundareson