Long gone are the days when holograms were the stuff of sci-fi movies and video games. Holographic technology is taking the medical world by storm, and by the looks of it, it’s here to stay. So how exactly is this technology helping doctors, and what are the major developments that we expect in the near future?
Back in 2016, Sopra Steria Norway acquired Microsoft’s HoloLens device to test potential applications of holographic technology in healthcare. What started as a test pilot in a few university hospitals quickly grew into the HoloCare consortium, a collection of initiatives across several countries that are developing holographic technology to help doctors deliver better care in fields as wide-ranging as surgery, tumor localization, medical training and remote therapy (or telehealth).
Understanding “mixed reality”
The first thing to understand about the HoloLens is that the technology underpinning it is different from virtual reality (VR). Whereas VR is about using a device to immerse the user in another digital world, augmented reality (AR) is about interacting with the real world, but with an additional layer of digital objects.
The HoloLens occupies a third and altogether new realm: that of mixed reality (MR). This means that a person wearing HoloLens can engage and interact with holographic objects in ways that were unthinkable only a few years ago.
The rise of x-ray vision, and more
In the medicine field, that interactive ability has led to some groundbreaking developments:
- Hands-free: While operating, surgeons can now keep their hands free as the hologram assists them with specific instructions, thus eliminating the need to ever manually check devices such as tablets or phones.
- 3D imaging: A team of medical professionals planning for surgery can use the HoloLens to project a 3D image of, say, an infant’s heart. They can observe its real shape and “zoom in” on anatomical relationships. This shared objective view means problems can be spotted sooner and potential solutions can be identified preemptively.
- Navigation: Before the HoloLens, surgical pathways could only be planned using 2D images that were overlaid on a map of the body. Now, holograms allow surgeons to literally place a map inside the patient’s body, enabling them to better navigate around certain tissue structures and avoid unnecessary damage.
- X-ray vision: Holograms can also be used to observe the status of organs or bones in real time. Take, for instance, a patient suffering from a bone malformation. Normally a doctor can’t see inside their body—they can only record the patient’s symptoms, meaning a lot of guesswork is still involved. Hologram technology allows you to actually see the bone structure and observe how it’s moving in real time. As one doctor recently said in an interview: “it’s like x-ray vision”.
Shift in communication
Beyond these immediate achievements, holographic technology will also completely disrupt the way medical professionals communicate on the job. Currently, medical science is built entirely around 2D imagery tools. So when students or doctors study the body, they use medical diagrams and communicate what they observe using words.
But language, as all codified systems, is imperfect. Holographic technology gives medical professionals a complete vocabulary of a disease or malfunction; a way of showing the specific problem without having to explain it with words—and thus of avoiding misunderstandings.
This change in communication will have huge consequences. Holographic tools will enable volumetric communication across different continents and timezones—features otherwise known as 3D Skype calls, or “holoportation”. This will transform the medical industry by eliminating doctors’ and patients’ need to travel.
Obstacles along the way
In many ways, mixed reality (MR) devices like the HoloLens are dependent on machine learning. While the ability of such devices to understand complex contexts will massively evolve in years to come, there is still some way to go.
Currently, devices like the HoloLens understand you’re in a room, but they do not understand the social context (are you by yourself in the room, where in the world in this room located, etc.)
When it comes to developing mixed reality devices, the key is therefore to simultaneously develop computers’ understanding of complex scenarios. And for that, we need to tackle two challenges involving data and design: more data to sync with the world and reconstruct it, and better design to enable users to interact with it in a natural and seamless way.
What the future has in store
Can holographic mixed reality devices conquer wider audiences and extend their influence beyond their specific applications in fields like medicine? As sci-fi author William Gibson once wrote: “The future is already here — it's just not very evenly distributed.”
While mixed reality devices are already available for purchase on the US market, they remain expensive and relatively niche. That’s because the difficulty in selling a VR headset or an AR device is that it’s not an iteration of an existing need—you’re creating a new and unfamiliar experience, which people need to be coaxed into trying themselves.
When you’re competing against other devices for entertainment experiences, such as smartphones and game consoles, the entry barrier is therefore quite high.
That might change, however, as perceptions shift. As future generations move beyond the conception of digital as “fake things on a screen,” digital objects will increasingly be treated as yet another material, similar to plastic, wood, and metal.
That reality is closer than many of us currently think.