Extended reality: what is XR?

A Pokémon that seems to run across the street in front of us on our screen? A 360° viewing of a property available for sale in another country? A sketch book with characters that appear in 3D after colouring them in? These are examples of extended reality. The umbrella term covers all immersive technologies that expand our real world and combine it with virtual elements. We explain what exactly extended reality is and what differences and similarities exist between the various technologies: augmented reality, virtual reality, and mixed reality.

The definition of extended reality – also known as cross reality (XR) – is any technology that creates computer-generated environments or objects. The term encompasses both developed and future forms of XR. The various technologies are differentiated primarily by their relationship between the real and virtual world. While augmented reality users perceive virtual objects as an extension of the real world, users of virtual reality immerse themselves in a purely virtual world. Another key aspect of the extended reality definition is that it always concerns an immersive technology.

Three forms of extended reality currently exist. We introduce these technologies as well as their differences and similarities below:

Augmented reality (or AR) expands on the real world by inserting virtual objects and information into the real world. Virtual elements include images, text, and animations. In the case of augmented reality, perception still centres around the real environment. Users are only able to interact with virtual objects to a limited extent – if at all.

Augmented reality is so far the most commonly used form of extended reality. One main reason for the success of this technology is the low technical requirement. Augmented reality can be used with just a smartphone, tablet, or PC, making it easily accessible to end users. What’s more, special AR glasses are also available that project virtual objects in front of the wearer’s eyes. Here, the external world remains visible.

A particularly well-known and successful example of augmented reality is the game Pokémon Go – available as an app for iOS and Apple. It is among the most successful games for mobile end devices and, having generated more than a billion dollars of profit, it is also the most successful AR game to date. With Pokémon Go, the mobile device displays digital characters (Pokémons) in the real world and thereby builds on reality. Extended reality has also arrived on social media channels: Filters allow users to add virtual objects like hats, sunglasses, or make-up to their photos and videos, for example.

Virtual reality (abbreviated to VR) creates a fully computer-generated 360° environment. Users completely immerse themselves into the virtual world and interact with it, while the real world is hidden from view.

In contrast to augmented reality, special devices are required for this XR or extended reality technology. Wearing a head-mounted display, users visually perceive the virtual world; the real world is fully concealed and invisible during use. The degree of immersion with this closed system is especially high and allows users to dive right into a virtual world. These devices usually require a computer connection. The best-known example of a VR headset is Oculus Rift by Facebook.

Mixed reality (or MR) works by combining elements of AR and VR. It’s the latest and most complex immersive technology. This form of extended reality combines our real world with virtual environments, thereby creating new surroundings. The user simultaneously interacts with the real and virtual environment. Physical objects in the real world have an effect on digital elements. The use of this technology requires an MR headset and a lot of processing power.

An example of a mixed reality headset is the Microsoft HoloLens presented in 2015. The headset transparently displays 3D objects in the user’s direct environment. The head-mounted display is equipped with integrated sensors, speakers, and its own computing unit. No additional equipment is required.

The following table shows the three current extended reality technologies in a comparison:

Find out more about the different types of XR – augmented reality and virtual reality – in this video:

Some augmented reality applications are already part of our daily life, such as the apps mentioned above like Pokémon Go or popular filters on social media. The use of virtual reality is advancing particularly in the gaming sector, but there are still technical requirements that stand in the way of mass adoption. By contrast, mixed reality is predominantly limited to companies. A number of challenges still need to be overcome for these latter two forms of extended reality to become common in everyday life. We outline the major hurdles below:

AR technologies like Pokémon Go or social media filters are accessible for any smartphone user. However, the use of virtual and mixed reality requires relatively expensive equipment. These immersive technologies are therefore primarily appealing to companies or specific target groups like gamers or very tech-savvy individuals.

Lots of detailed as well as personal data is often collected with the use of XR technology, including the user’s personal environment. For this reason, compliance with all data protection requirements is a challenge that developers need to face up to if they want to make their extended reality products accessible to a broad range of users.

During a Pokémon Go gaming event with over 20,000 participants, serious network problems arose. The event organisers were forced to refund all admission proceeds and accepted losses in the millions. This is a striking example of the fact that widespread use of extended reality also calls for IT infrastructure upgrades.

There is still development potential in terms of the technical devices themselves: For instance, the Microsoft HoloLens can be controlled with gestures and voice commands, but gesture controls are limited to just two hand movements. Moreover, head-mounted displays are often too heavy and uncomfortable for prolonged use. For end customers, the design is also an important purchase factor which manufacturers frequently give insufficient consideration to.

The extended reality market is experiencing constant growth. So far, it has typically been the entertainment industry at the forefront, where most investment is made in developing and distributing the technology. But XR and its versatile applications are becoming increasingly relevant in other sectors – such as in the medicine, military, industrial, and tourism industries.

In the entertainment industry, the best-known application of extended reality is video gaming. This is also due to the fact that the gaming industry adopted VR technology relatively early on, making it available to end customers. But the new technologies are promising many new opportunities for events like concerts and sports competitions. Participants could visit events from the comfort of their home via an internet connection and still experience a real concert or theatre performance thanks to extended reality and a virtual environment. Event organisers would then have the possibility to increase their ticket sales considerably.

Existing and prospective customers can discover and try out products in a virtual environment with the use of XR. In addition, this technology offers companies new approaches for interacting directly with customers remotely.

Extended reality offers significant advantages for companies and customers in this sector: Prospective buyers or tenants of an apartment or house can conveniently view properties from their own home. This reduces the workload of real estate agents and owners. When planning a new property, the immersive technology lets architects and designers visualise their ideas.

Extended reality enables companies to let potential customers try out their products. Here, immersive technologies represent a significant opportunity for online stores. When online shopping, customers are normally unable to see and touch products before they buy. At least the former is now made possible with virtual counterparts thanks to the use of XR. Learn more about the use of augmented reality in e-commerce in the article “Augmented Reality in E-Commerce”.

Extended reality also offers added value for customers in brick-and-mortar retail. The furniture store IKEA is already using this technology – in the form of augmented reality. Using the corresponding smartphone app, customers can project items of furniture into their own home. This shows what a selected piece of furniture would look like in their home, easing the purchase decision.

From the military and medical sectors to aerospace: With many professions, it is difficult or even impossible to train for risky situations without exposing people to danger. Thanks to XR technology, trainees are able to train their behaviour and actions in such situations within a safe environment, allowing them to prepare accordingly.