Technology is transforming the world around us. It enables us to do amazing things. However, what if we could go further and blend the digital world with the real world? Augmented reality (AR) promises to help us do that. AR can change the way we visualize, share ideas, learn, and imagine. It allows the addition of visual overlays or 3-D digital imagery on the real (physical) world and enhances situations in the real world with information from the digital world. The enhanced version of the reality can be viewed through a wearable termed as a Head Mounted Device (HMD).
Augmented Reality is commonly confused with its close cousin Virtual Reality (VR). While the advanced versions of both AR and VR headsets provide stereo 3D high definition video and audio, they differ in concept and use cases. AR is open and partly immersive - a user can see through and around it. Put simply, VR is closed and fully immersive and puts the user inside virtual worlds. On the other hand, AR places virtual objects into users' real worlds; thereby, augmenting them. Think of it as the character from the movie Ironman™ (AR) vs. the one from Avatar™ (VR). While the character of Ironman™ in the movie sees, additional information displayed to him in his helmet and get voice prompts as he goes about doing his business, the characters in the movie Avatar™ live completely different lives in the virtual world while their physical being stays back at a different place. The industry is still scratching the surface as far as the commercial uses of AR is concerned. The biggest limitation today is simply the lack of suitable content, which needs to be created from the ground up. However, Augmented Reality got a big push with Apple’s announcement of ARKit™ during Worldwide Developers Conference (WWDC) 2017 in San Jose. ARKit provides a set of tools for developers to build Augmented Reality into their applications on iOS 11. Apple’s ARKit™ will compete directly with Google’s Tango™, which is already available in the market. Just like with smartphones, we expect the development of complete platforms for AR with some of the heavyweights such as Apple and Google already making headway.
Some interesting use cases of AR include:
- Retail: German retailer Saturn™ is trying out Augmented Reality sales staff called Paula™. If you enter the retail store wearing a Microsoft HoloLens™, Paula™ can guide you to a selection of products and can tell you their features. It can also assist in the sales process.
- Education: AR is enabling learning through real-life simulations in place of highly sophisticated teaching equipment that would have been necessary for a complete demonstration. AR can bridge the gap between the virtual and physical worlds. AR applications and platform enable students to visualize 3D models in the real environment and in real time, helping them better understand the concepts they are trying to learn.
- Manufacturing & Maintenance: Airbus™ is using wearable technology to accelerate the labor-intensive aircraft maintenance process significantly with improved precision. Aircraft maintenance is very difficult because knowing the tools, parts, and instructions take years of training. When doing maintenance, employees can put on a head-mounted display and visualize the complete activity, potentially reducing the training time.
- Construction: AR applications could allow teams to visualize what the finished project is going to look like at the construction site even before the construction begins. An architect can enter a house and double-click to change color or layout and see that immediately.
- Tourism: You can learn more about a city’s landmarks or museums at your own leisure as you walk through them. With AR, tourists can have access to up-to-date information relevant to their needs and preferences. A big problem for tourists travelling internationally is their inability to read or speak the local language. Even understanding directions could be difficult in certain cases. AR can translate different languages by just scanning the language displayed at signage, airports etc.
The AR Headset System
An AR headset even in its simplest form needs to recognize the environment, understand gestures, process the information, and project the digital content in real-time. A forward looking high resolution camera fitted to the HMD helps to capture the environment in the field of view (FOV) while a series of sensors including depth sensing cameras help identify the spatial relative positions of various objects. The optical system in an AR headset is very sophisticated as the user needs to be able to see the world around through the lenses. After processing the information from sensors and cameras, an optical projection system beams the digital information into the user’s FOV.
While some of the early designs of AR headsets such as the Google Glass aimed to project digital information to help the user make better decisions, some of the newer and more advanced AR headsets such as the Microsoft HoloLens pack in a lot more processing capability, better optics and power management to allow the user to interact with these digital objects. All the capability of these advanced headsets allows the user to see a virtual object in 3D and ‘work’ with it as if it existed in the real environment.
Memory Choices in AR Headsets
AR headsets generally are untethered devices which allow you to operate with your hands free. This also means that much of the computing happens right on the headset. The early versions of the Google Glass sported a TI OMAP 4430 SOC (System on Chip), 1GB (gigabyte) of Low power DRAM, 16GB of storage, a 5 MP (mega pixel) camera and Android 4.4. The first generation of the Microsoft HoloLens device however is powered by an Intel 32-bit architecture, GPU and a custom built holographic processor (HPU) with 2GB of mobile DRAM (LPDDR) and 64GB of storage (eMMC). Given the type of applications that are enabled by these AR headsets which includes but are not limited to the ability to communicate with people many miles away, we should expect these advanced AR headsets to be architecturally like smartphones in the near term. The processors driving the computation in these headsets are more likely to resemble chipsets in high-end smartphones (high performance and efficient power use) than the processors used in PCs and servers. In addition, storage in the form of eMMC, SD cards, or SSDs will be necessary to store data locally on the headset. Various SLC/MLC NAND options, LPDDR2/3/4 options, eMMC, or Multi-Chip Package (MCP)) options that combine NAND and LPDDR in the same package for effective integration and space savings could serve the memory needs of such untethered AR headsets.
What the future holds
Although AR and VR devices are separate devices today, we can foresee a future in which there will be a single device accomplishing both tasks. As this blog mentions, both AR and VR lie on a reality continuum with varying degrees of digitization. As resolution of the displays improve and high quality, low latency rendering of data becomes a requirement, computing needs of these headset may exceed the capabilities of advanced smartphones. It’s possible that such capabilities are accomplished with dedicated hardware and software that is more efficient in addressing the end application.
Additionally, the advancements in AR is happening at an interesting time and running in parallel to advancements in Machine Learning (ML). Simply said, ML is the technology which gives computers the ability to learn without being explicitly programmed. In ML, image recognition and speech analysis are two of the most prominent workloads today. This also aligns well with the capabilities of an AR headset. As ML makes progress, AR headsets can potentially serve as intelligent end devices which could be used to train these ML algorithms. Smarter and more intelligent algorithms in turn can help to serve the most relevant information to AR glasses and make them indispensable devices.
Summary
Augmented reality is an exciting new market that is evolving at a remarkable pace. As the concept of a connected world takes shape and develops into reality, augmented reality glasses will allow humans to interact with the world around them but also with each other. Its early stages today with a long exciting runway ahead for this technology. Micron will continue its efforts to support and enable augmented reality headset as a product category with memory products that will not only differentiate the product, but will also ultimately enhance the experience for customers.
References
