ANY SUFFICIENTLY ADVANCED TECHNOLOGY IS INDISTINGUISHABLE FROM MAGIC
-ARTHUR C. CLARKE
Before we get going into how AI is changing the reality, here’s a quick overview of AR/VR/MR:
Augmented Reality – digital information overlaid on the actual world of reality
Virtual Reality – Full digital representation of our reality
Mixed Reality – A combination of virtual reality and reality
Stanley Weinbaum in his short story Pygmalion Spectacles introduced the world to the idea of goggles that allow users to experience an artificial reality. Weinbaum pondered over this idea and went as far as creating a reality that also has taste and touch along with the vision. Though works of Weinbaum were considered to push the field of fiction away from the doldrums of the early 20th century, the idea of this reality within reality was still considered a bit of a stretch.
However, today, after decades of innovations, we have developed wearable devices that transfer us to an almost life-like reality where one not only gets to ‘see’ but also ‘be’.
As the efforts were being put into making AR/VR devices accessible, making machines to think also have gained traction at the same time. Today, at this intersection of interdisciplinary associations, artificial intelligence is helping to make virtual reality more real.
Currently, Facebook’s Oculus is one such endeavor that has been at the forefront of bringing AR and VR devices to the fore. Oculus Insight marks the first time that fully untethered six-degree-of-freedom (6DoF) headset and controller tracking has shipped in a consumer AR/VR device. Built from the ground up, the Insight stack leverages state-of-the-art computer vision (CV) systems and visual-inertial simultaneous localization and mapping, or SLAM.
How AI Improves Oculus?
Facebook Reality Labs (FRL) has in-built virtual spaces that allow AI researchers to train a model to understand digital spaces. For example, one such task can be to find missing keys in a life-like digital living room.
The accuracy of these approaches relies on a combination of a well-engineered and integrated camera rig with a high-accuracy depth capture system, a state-of-the-art simultaneous localization, and mapping (SLAM) system, and a dense reconstruction system.
Research is being done in these domains for decades and many techniques were experimented with. Of which, SLAM techniques are one. However, the technology has only recently become mature enough for consumer applications, such as driverless cars and mobile AR apps.
AI can help solve the challenges of rendering highly compute-intensive visuals in VR.
DeepFocus is aimed at enhancing VR images, relies heavily on deep learning. It constitutes a novel end-to-end convolutional neural network that produces the image with an accurate retinal blur as soon as the eye looks at different parts of a scene. The network includes new volume-preserving interleaving layers to reduce the spatial dimensions of the input while fully preserving image details. The convolutional layers of the network then operate on the same, reduced spatial resolution, with significantly reduced runtime.
To establish the location of objects in relation to other objects within a given space, these current systems use a custom hardware architecture and advanced computer vision algorithms — including visual-inertial mapping, place recognition, and geometry reconstruction.
The rise of smartphones with high-density displays and 3D graphics capabilities has enabled a generation of lightweight and practical virtual reality devices. The video game industry has continued to drive the development of consumer virtual reality unabated. Depth sensing cameras sensor suites, motion controllers and natural human interfaces are already a part of daily human computing tasks.
Recently, companies like Google have released interim virtual reality products such as Google Cardboard, a DIY headset that uses a smartphone to drive it. Companies like Samsung have taken this concept further with products such as the Galaxy Gear. Over the last 40 years, personal computers, smartphones, and tablets have given us constantly, near-instantaneous access to the digital world through 2D screens, in the process touching almost every aspect of our lives.
The team at Oculus believe that the next 40 years will see AR and VR radically change the way we work, play, and communicate. As more research steers towards mixed reality, the reach of AR/VR is turning speculation to reality. And, what new avenues it will uncover is waited to be seen.