Reality Technologies

May 3, 2017

Last week, I attended Into the Future(Part - II): Transmogrified Reality Meetup in San Jose organized by @techinmotion. Even though I had collaborated on a project that involved Augmented Reality, I would consider myself to be a novice in this fast evolving technological landscape that is attracting a lot of attention. The intent of this post is to help explain Reality Technologies, the vendors, and the use-cases that they are addressing.

Virtual Reality

As the name suggests, Virtual Reality provides an immersive experience in a make-believe or fictional or fantasy world that is completely different from the real world. It is important to understand that the immersion is achieved by deceiving the user’s senses of sight and sound to believe that they are in a different setting/environment. These experiences are achieved through a head-mounted display(HMD) along with a tethered PC with an extremely powerful graphics cards or GPUs.

Most of the Virtual Reality headsets resemble ski goggles and use a pair of screens displaying two images side by side, one for each eye. The 3D effect is created by placing a set of lenses on top of those screens. The embedded sensors pickup the user’s head movement and the image is adjusted accordingly.

Here are some of the VR platforms available in the market:

Facebook’s Oculus Rift

PIMAX 4K UHD

HTC Vive

Currently, the use-cases that are gaining traction with Virtual Reality include rich interactive video games, visual learning experience, and training. Lack of content and compelling reasons/use-cases for one to be glued to a Virtual Reality headset/goggles for an extended period of time during a day are the reasons why some people think that Virtual Reality is for a niche audience and may not materialize into a huge market. However, with significant investments tied into this technology from some of the major companies along with VR experience such as Capturing Everest becoming available, I would think that Virtual Reality is going to be around for a while.

Since the number of Virtual Reality platforms are increasing, it is also becoming a challenge for application/game developers to port their applications and games to each vendors’ APIs as the VR device can only run apps that been ported to its SDK. To address this fragmentation, a cross-platform architecture called OpenXR is being developed to enable applications and games to be written once to run on any VR platform.

Augmented Reality

Unlike Virtual Reality, in Augmented Reality we are dealing with the real world that has been modified or enhanced with contextual information by superimposing graphics over a view of the real world. In Augmented Reality, the user’s senses will not be deceived but continue to function in a real, albeit a bit more richer and interactive surrounding. For example, in a smartphone’s camera held against a window, you can see planes headed in the direction but are not visible to the naked eye at that point in time. Or, you can look at the temperature and the weather data on your bathroom mirror.

Consumers can get the taste of Augmented Reality technology using either smartphone apps or untethered transparent glasses such as Google Glass. Here is a youtube video for Google Glass and it’s features:

Some of the popular use-cases that feature Augmented Reality are:

Pokemon Go

IKEA Concept Kitchen

IKEA Furniture

Mixed Reality

Mixed Reality uses the real world as the canvas and overlays it with elements from virtual world to create new environments made up of physical and virtual objects that can interact. Mixed Reality keeps the real world interactivity aspect of Augmented Reality and merges it with the fantasy aspect of Virtual Reality. The fact that the objects from the real world and the objects from the virtual world can react to each other in real time makes Mixed Reality very compelling.

With the HoloLens initiative, Microsoft is betting big on holographic computing as it opens up Windows Holographic operating system to VR headset makers to push it’s vision of Mixed Reality where VR headsets interact with the HoloLens to create holograms in the real world.

So, what is is a HoloLens? In simple terms, it is a special headset that has the ability to let a user interact with holograms in a real world space such as a living room or a board room. Unlike the VR headsets which are tethered, these headsets are untethered and can communicate wirelessly with a PC and use high-definition lenses for 3D display and spatial sound technology to create an immersive yet interactive holographic experience.

Microsoft has put out some really exciting videos showcasing HoloLens like the one below that includes holoportation as one can interact with holograms of family members, who are in a different part of the world, in one’s living room:

Or, play Minecraft on a table:

Windows 10 will include APIs for developers to create holograms to make holographic experiences for the HoloLens.

Microsoft is working with NASA, car manufacturers, school districts, construction companies, gaming companies, etc. to help increase the adoption of HoloLens across various segments of the economy.

Transmogrified Reality

I hadn’t been exposed to this term till I signed up for the aforementioned Meetup. While Virtual Reality and other forms are exciting, they require the users to strap on headsets for long periods of time which can be fraught with risks. Google’s, Game Design Chief, Noah Falstein describes Transmogrified Reality as the most realistic form of virtual and augmented reality conceivable through our current and emerging technologies. Based on the demos in Noahs’ Game Developer Conference 2015 presentation, one can surmise that consumers will be experiencing Transmogrified Reality via Google’s Project Tango tablet. With real time 3D-sensing abilities and inexpensive head mounts, Tango will be able to capture it’s vicinity or the surrounding area and incorporate elements of virtual reality into a player’s vision while allowing the player move freely around any environment in the real world.

Here is Noah’s presentation where he talks about Transmogrified Reality:

Tango

In conclusion, VR/AR/MR/TR are all incredibly exciting technologies that have tremendous potential in not just gaming/entertainment but various other industries such as automotive, healthcare, education, space, tourism, marketing, retail etc.