Picture this: A surgeon operating a robotic scalpel refers to a 3D model of a human body in minute detail to plan her next snip. She gestures to enlarge the model that is projected in front of her in mixed reality.
Welcome to the exciting world of augmented reality (AR) in healthcare and surgery, where scientific fantasy meets life-saving innovation. In this brave new realm, augmented reality not only augments our understanding of the human body but also revolutionises the way surgeons operate, potentially leading to faster, safer and more precise procedures than now.
The advent of AR in the surgical theatre follows another futuristic technology, that of a robo-surgeon. When AR meets robotic surgery, the result is an unprecedented amount of information and control for surgeons.
“The AR headset [that surgeons wear] allows them to see all scans and pre-surgery planning that were uploaded before the surgery. You just move your eyes around this visual field — left, right, up and down — and you have everything. It’s like a PlayStation,” says Dr Cezar José Mizrahi, who is among the few surgeons to be familiar with both modalities.
The first combined surgery
In August, Dr Mizrahi performed the world’s first surgery to pair both AR and robotic surgery at the Shaare Zedek Medical Center in Jerusalem. The case was that of a 25-year-old who presented doctors there with a severe fracture in the lower thoracic region of his spine.
Not just Dr Mizrahi, but the robo-surgeon that had become a part of the clinic over the last few years also had to be familiarised with the individual needing care. “The robotic arm achieves a three-dimensional recognition of the patient based on previously done scans. It achieves ‘perfect registration’, which gives surgeons the ideal trajectory for their instruments,” Dr Mizrahi shares with Happiest Health.
The surgery was critical — without it, the individual had a high risk of further neurologic deficit and could even become unable to move the legs. But also critical was the need for precision — such surgeries have significant risk such as a screw accidentally being placed on the spinal cord or nerve roots. In such situations AR minimises those risks.
Comprehensive view
The AR headset, a pilot technology by healthtech company Medtronic, gave Dr Mizrahi and unprecedented amount of information to work with during the surgery. With an overlay of the individual’s anatomy before him, Dr Mizrahi could freely view a model of the person’s body from the top, side and front. The AR headset also let him view all past CT scans of the person, a lineup of the surgical screws to be inserted, and even a magnified view of the site of surgery.
“The experience helped me perform the procedure exactly as I planned it, and only make adjustments that I might have felt necessary as the surgery developed,” he says.
Within a day of the robo surgery, the 25-year-old walked on his own and after five days, he was discharged.
The Indian application
In 2022, All India Institute of Medical Science, New Delhi (AIIMS), tied up with US-based ImmersiveView to make use of the latter’s VR headset and technology. By viewing CT, CBCT, 3D angiograms, and MRI scans with it, the doctors can create a digital twin of a person’s anatomy.
Where AR can really shine will be in representing the body in 3D, notes Christian Moro, associate professor of biomedical sciences and medicine, Bond University, Gold Coast, Australia.
“The way our organs and systems are shaped, and how they are structured, is very hard to justly represent in 2D planes. Providing patients and students with images and models in 3D really helps with forming a deeper understanding,” he says.
Study anytime and anywhere
While real specimens and models will continue to have their uses, AR can help reduce the risk of making mistakes, and can allow one to study at any place in any time, he notes.
The best part about augmented reality could be that most people already have the most important device needed for it — a smartphone. “Most phones have a quality camera and extensive processing power, and can facilitate excellent experiences in AR,” Moro notes.
AR is already becoming an essential tool in diagnostics and both robo surgery and human surgeries. “The use of computer data and computer imagery overlayed during real-world scenarios is a fantastic merging of the digital and physical spaces,” he says.
No substitute to a surgeon
As with AI, AR is also seen as a supplement, not a replacement, to a doctor. Even surgeons who have pioneered the use of tools are certain that they will not replace them.
“In my opinion, AR will be [yet] another tool in the surgeon’s arsenal to achieve better results to our patients. I still do not see robotics, AR or any other technology as a replacement for the surgeon,” says Dr Mizrahi.
Moro concurs. “There is always the risk that we can place too much trust in technology and the algorithms. While technology can assist with a few of the stages, it remains optimal to have a skilled and experienced professional at the start and the end of the healthcare journey.”
