Creating believable Virtual Reality (VR) experiences has gotten quite easy over the past four years. Augmented Reality (AR), on the other hand, is quite a bit more difficult to achieve technically, creatively and ergonomically.
Let's start with my definition of VR vs AR as there can be some grey areas betwixt them. For me, VR is any experience that replaces the real world with an artificial one. Be it camera-based content, or computer generated content, the VR user does not make eye-contact with real people and objects around them. In contrast, AR is a layer of content over top of the real world. You would be able to make eye contact with people and objects and environments. Specifically, I want to talk about AR that uses some form of a head-mounted display or HMD; I'm not discussing the old "magic window trick" you get with AR experiences or games like Pokemon Go.
AR is a very promising panacea. It's also the least anti-social form of computer-assisted immersion experiences. There are several technical hurdles in the way of really good AR experiences that need to be addressed. Here's my short-list of hurdles I see in AR tech. I'll explain each below. Inside-out tracking, black-point, occlusion, Field of View (FOV), and wearability.
Although there's still room for improvements, inside-out tracking is well on it's way to not being a show-stopping hurdle for AR anymore. Microsoft's Hololens does markerless positional tracking quite well. Inside-out tracking means the HMD device is tracking your position. Today's high-end VR use outside-in tracking - I expect that will change soon. For systems like the Oculus Rift and HTC Vive, a camera or laser grid system that is separate from the user's head-mounted display (HMD) is required to track the user's position in the VR space. For AR on HMD, specifically untethered wireless AR systems, that positional tracking requirement needs to come from the HMD itself. Indoor environments will not have access to GPS data for position, inside-out tracking will be a required technical hurdle to master. Data that is captured from normal and infra-red camera pairs can be used as a means to understand the user's space, processing that data quickly, accurately and knowing when to keep or discard or optimize that data will limit developers on what can be done, and how.
Black-point is a big deal for rich colors and high-contrast images. It's a similar problem that projectors have. Electronic images projected onto a wall or the back of your eyeball can only be a dark as the darkest color in the room. Projected electronic content does not create darkness, only light. So, if you're in a gray room your darkest color will have the luminance of the gray room. Turning off the lights is not an option for practical AR technology to succeed. Magic Leap, a stealthy player in the AR space, supposedly has a solution for black-point. Presumably, the technical hurdle for AR is creating an opaque, or semi-opaque pixel for the eye yet still capable of full transparency when needed. And doing that at high-resolution and high frame rates.
Occlusion is another area that will be challenging for AR. Occlusion, in this case, means knowing when augmented electronic content is supposed to be behind of real-world Object A, but in front of real-world Object B. It's a likely a matter of capturing high-quality depth information and should be addressed with the inside-out tracking solutions, but making that illusion seamless and believable will require a lot of data to be stored or processed. So, assume mediocre results for a few more years.
Field of View is a big deal in VR. The more FOV you have, the more practical problems you have to solve, but large FOV means greater immersion. It's a whale worth chasing for VR hardware manufacturers. For AR, a deep sense of immersion is not as much of a requirement for most applications, but situational awareness of augmented content that can be seen in the periphery will be something most people should expect after experiencing it in all the VR systems. If you've haven't tried the Hololens, the FOV is about the size of a medium-sized post-it note held out about 2 feet from your eyes. That means if you're in a room of augmented content, whatever it is, you only perceive what is directly in your line of sight. Mo FOV, mo problems. The solution to this sort of AR display requirement doesn't exist in any practical format yet.
It could be that eye-tracking and low-rez proxy images reduce some of the technical issues for FOV in both AR and VR, but that will also mean a bulkier HMD requirement for AR. This leads us to the very practical wearability requirement hurdle. With the exception of very specific uses and applications where the benefit of AR exceeds any of it's encumbrances, the general population use of AR HMD's will need it to be comfortable, fashionable and almost invisible. If anyone recalls the nerdy pioneers that tried to wear Google Glass in public, it's clearly a technology most other people didn't want to see. AR will take a few more technical and practical miracles before it will valuable enough to outweigh the inconvenience and social stigma quotient.