A short while ago, Tested posted a video all about hands-on time with virtual reality (VR) headset prototypes from Meta (which is to say, Facebook) and there are some genuinely interesting bits in there. The video itself is over an hour long, but if you’re primarily interested in the technical angles and why they matter for VR, read on because we’ll highlight each of the main points of research.
As absurd as it may seem to many of us to have a social network spearheading meaningful VR development, one can’t say they aren’t taking it seriously. It’s also refreshing to see each of the prototypes get showcased by a researcher who is clearly thrilled to talk about their work. The big dream is to figure out what it takes to pass the “visual Turing test”, which means delivering visuals that are on par with that of a physical reality. Some of these critical elements may come as a bit of a surprise, because they go in directions beyond resolution and field-of-view.
At 9:35 in on the video, [Douglas Lanman] shows [Norman Chan] how important variable focus is to delivering a good visual experience, followed by a walk-through of all the different prototypes they have used to get that done. Currently, VR headsets display visuals at only one focal plane, but that means that — among other things — bringing a virtual object close to one’s eyes gets blurry. (Incidentally, older people don’t find that part very strange because it is a common side effect of aging.)
The solution is to change focus based on where the user is looking, and [Douglas] shows off all the different ways this has been explored: from motors and actuators that mechanically change the focal length of the display, to a solid-state solution composed of stacked elements that can selectively converge or diverge light based on its polarization. [Doug]’s pride and excitement is palpable, and he really goes into detail on everything.
At the 30:21 mark, [Yang Zhao] explains the importance of higher resolution displays, and talks about lenses and optics as well. Interestingly, the ultra-clear text rendering made possible by a high-resolution display isn’t what ended up capturing [Norman]’s attention the most. When high resolution was combined with variable focus, it was the textures on cushions, the vividness of wall art, and the patterns on walls that [Norman] found he just couldn’t stop exploring.
Next up at 39:40 is something really interesting, shown off by [Phillip Guan]. A VR headset must apply software corrections for distortions, and it turns out that these corrections can be complex. Not only does an image get some amount of distortion when passing through a lens, but that distortion changes in nature depending on where one’s eye is looking. All of this must be corrected for in software for a high-fidelity experience, but a real bottleneck is having to wait for a physical prototype to be constructed, and complicating this is that different people will have slightly different subjective experiences of distortion. To address this, [Phillip] shows off a device whose purpose is to accurately simulate different physical headset designs (including different lenses and users) in software, allowing exploration of different designs without having to actually build anything.
The final prototype — named Starburst for reasons that will soon become clear — is showcased at 44:30 and demonstrates the power of true high dynamic range. It’s the most unwieldy-looking, but that’s mainly due to essentially having car headlamps as backlights. The purpose isn’t to blind users, but to deliver something important, and lacking. Why is high brightness so important? The answer is simple: light levels in the real world are far beyond anything a modern monitor (or VR headset) can deliver. This means that, in VR, a spotlight only ever really looks like a picture of a spotlight. It will never truly look bright, not in the way that your eyes and brain actually experience the word. When headsets can deliver a true HDR experience, that will change, and that’s what this prototype delivers.
It’s clear this direction is being taken very seriously, and it may come as a surprise to learn that delivering a convincing visual experience goes considerably beyond higher resolution and wider field-of-view. All of the truly good VR ideas may have been dreamt up back in the 1960s, but this video is a great showcase of what goes into the nitty-gritty scientific work of figuring out how to get a problem solved.