The New Wave of Imaging Technology in Medicine
Jayender Jagadeesan, PhD, and the National Center for Image-Guided Therapy explore the ways augmented, mixed and virtual reality can be used in medicine.
From the discovery of x-rays in 1895 to the utilization of ultrasound, MRI, and CT scanners in the 1960s and 70s, health care has long depended on imaging to provide the best quality diagnostics and care for patients. Now, as new forms of imaging such as virtual reality and augmented reality hit the scene, researchers including Jayender Jagadeesan, PhD, of the Surgical Planning Lab in the Radiology Department at BWH, look to bring these imaging techniques into the operating room.
Immersive technologies mix the real world with digitally simulated environments to transport the user into a virtual, augmented or mixed reality. Though all slightly different, each type of “reality” opens the door for brand new applications in medical imaging.
“There is a lot of information and new technology available for physicians,” said Jagadeesan. “The question is, how do we utilize all of these images and information in real-time during surgery without going into information overload?”
Enveloped in a Digital World
image 1
As the most immersive type of imaging, virtual reality (VR) takes the user into a completely digital world with digital objects. Jagadeesan and his colleagues at the National Center for Image-Guided Therapy (NCIGT) at BWH see virtual reality as a potential tool for use in diagnostics and education. The NCIGT is an NIH-funded center – led by Clare Tempany, MD – whose objective is to enhance medical procedures using patient-specific imaging during surgery or therapy. In radiology, it is difficult to look across several scans to spot minute changes in small masses in the lungs over time to determine if they are cancerous. But with a wrap around VR headset such as the Oculus Rift, a radiologist could project multiple images simultaneously across their vision, making it easier to identify changes. VR could also allow physicians to collaborate on diagnostics from remote locations. They would be able to view the same scans at the same time, and use a digital rendering of their hands to point out various areas on the scans (image 1). This shared virtual reality could also be used as a tool to educate new physicians.
image 2
With mixed reality (MR), another form of immersive technology, the user sees digital objects superimposed onto a digital rendering of the real world. At the NCIGT, researchers are testing the use of MR in laparoscopic surgery. Researchers combine information from webcams, the laparoscope, the navigation system and 3D pre-operative imaging and make the information all visible through the Oculus Rift (image 2). So far, they have used MR only in pre-clinical studies to demonstrate the feasibility of using mixed reality in laparoscopic surgeries, but they hope to begin using it for lymph node removal surgeries in collaboration with Ali Tavakkoli, MD, later this year.
Clinical teams are also exploring applications for augmented reality (AR), the process of taking digital images and overlaying them on top of the real world. With the popularity of games like Pokémon Go, most smart phone users are already familiar with this kind of imaging. Augmented reality could be useful for surgical planning prior to an operation. Using the HoloLens, a wearable, head mounted AR device, surgeons can generate 3D models of the patient’s anatomy to create a plan for the best approach to surgery (Images 3/4).
images 3/4
Expanding Uses
In the future, Jagadeesan and the NCIGT are looking to use the HoloLens for spinal tumor ablations (with Tom Lee, MD), skull-base surgeries (with Eduardo Corrales, MD), neurosurgeries (with Alex Golby, MD) and sarcoma resections (with Raut) and may start recruiting patients in the near future.
The future of immersive technology in medicine is continually expanding. “This is just the beginning of what could be possible with these new tools,” said Jagadeesan. “Enhancing our reality with immersive technologies is an exciting and promising way to further advance what’s possible in medicine.”