A Real Discussion on Augmented Reality Surgery

In the relentless pursuit of enhancing patient outcomes using advanced technology, surgeons have been drawn to the promise of augmented reality (AR) surgery. The prospect of utilizing cutting-edge technology in the operating room is met with enthusiasm, but a critical question lingers: Has augmented reality surgery advanced far enough to be reliable?  

Delving into the roots of AR navigation, we uncover its evolution from Ivan Sutherland's groundbreaking proposal to its current status as a transformative force in various medical specialties.  

Has Augmented Reality Surgery Advanced Far Enough to Be Reliable? 

Many surgeons are excited by the concept of augmented reality (AR) surgery, but they want to know that it has been tested and proven first.  

Augmented reality surgical navigation was first proposed to the U.S. military by Ivan Sutherland in 1965. This innovative system comprised a head-worn display and an image generation subsystem. However, due to its unwieldy, heavy, and expensive nature, the technology wasn’t adopted into the medical field.  

Since then, leaps and bounds have been made in the field of AR, but it has only been in the last 15 years that computer software, hardware, and processing speeds have advanced far enough to begin supporting the technology required to run affordable AR-based surgical simulators.  

The technology is becoming increasingly versatile, cost-effective and comfortable, and now AR is being used in spine surgery, neurosurgery, plastic surgery, interventional radiology, orthopedics, and more. Over fifty companies are currently in the race to make AR the new norm in the OR, and the industry is expected to grow to $7.42 billion by 2027. 

AR navigation is being embraced by doctors and medical professionals around the world. Its applications are expanding, and its integration in surgery is proving to yield superior results in certain procedures when compared to freehand and conventional methods.  

Is AR Surgical Guidance Accurate? 

The reliability and accuracy of AR surgical navigation is determined, in part, by methods of calibration and registration, and in the last decade these methods have advanced substantially.  What methods are currently in use, and which are most accurate? 

Calibration 

For an industry-leading AR solution, there are three types of calibration: room, eye, and instrument. Other solutions would have part or all these as well. The requirement for any system is to accurately assess the layout of the room, assess the location of an instrument in 3D space, and adjust for variance in the users interpupillary distance. 

Room Calibration involves the recognition and adjustment of the AR system's position within a room. Sensors play a crucial role in establishing a coordinate system for the AR headset to operate within. 

Eye Calibration is essential for ensuring that virtual images displayed by the AR system align accurately with the real world as perceived by the user's eyes. The positioning of the user's eyes behind the AR lenses can impact this alignment, particularly in navigation scenarios where visual matching of real instruments with virtual surgical guides is required. 

Instrument Calibration is the process through which an AR system precisely tracks the location, orientation, and length of a medical instrument. This typically involves attaching a fiducial to the instrument and running a calibration procedure to acquire the necessary parameters. For example, this calibration enables the AR navigation system to inform the surgeon if the needle-tip is correctly positioned in terms of both location and depth within the patient's body. 

So, what systems can re-calibrate seamlessly on the fly? The Microsoft HoloLens 2 has been the system of choice for some AR surgical companies, in part due to its advanced capabilities in calibration.  

Novarad’s VisAR system uses the calibration provided by the Microsoft HoloLens 2 and enhances it. The accuracy is more precise than the device alone can provide. 

Registration 

The overall precision of AR surgery hinges in part on the choice of registration methods. Let's delve into the four specific registration types employed by AR systems. 

1. Surface Mesh Triangle Registration: This is one of the earliest methods of registration and is still commonly employed by many AR systems. It creates a mesh of triangles over the surface of everything in view. While it does provide accuracy, it is typically far less accurate than other methods. 
2. Manual Registration: Surgeons manually position and rotate images to fit the patient using gestures and dragging movements. Accuracy depends on the surgeon's skill and the visibility of features on the patient's contour. 
3. Landmark Registration: This approach involves pinpointing anatomy on the patient using a calibrated instrument. The identified points of interest are then matched to corresponding points on medical imaging data. 
4. Fiducial Registration: Utilizing a physical item attached to the patient during both the scan and procedure, this method can offer dynamic updates when the patient moves. This is the most flexible method and provides simple workflows. Fiducial registration in systems like VisAR can be scanned both pre-operatively and intraoperatively. 

VisAR augmented reality surgical navigation favors fiducial registration for its live feedback capability and increased accuracy, achieving +/- 1 mm accuracy in bone/brain areas, making it the preferred choice for procedures. 

Is AR Surgical Navigation Legal & Approved by the FDA? 

Over the past decade, there has been a substantial rise in the adoption of AR technology. Currently, its primary medical applications are in spine surgery and neurosurgical education. It is particularly prominent in North America, Europe, and China. Other countries throughout the world such as Indonesia, Vietnam, and Mexico are also readily adopting AR in surgery.  

Not only is immersive AR surgical navigation legal, but many solutions are FDA cleared for use and it is proving to be superior to traditional surgery. We have had many surgeons express surprise at its ease and accuracy.  

The FDA is taking a very close look at this technology and steadily providing clearance. Through meticulous and data-driven testing, multiple procedural augmented reality solutions have gained approval so far, including products such as VisAR, MediView XR, Augmedics xvision, and Medivis SurgicalAR.  

VisAR obtained 510(k) FDA approval for stereotactic spine surgery in 2022, and is currently pending in other specialties.  

Is AR Surgical Navigation a Valuable Tool for Surgeons? 

There are many highly skilled surgeons out there. The education and training of surgeons has never been better, and miraculous procedures are performed every day now.  Augmented reality tools are not intended to replace a seasoned professional, but simply enhance their abilities.  

The preoperative and intraoperative features of AR enable a surgeon to advance their procedures to the next level, completing surgery faster, smarter, and with less invasive methods. AR surgical guidance enables a surgeon to view presurgical plans immediately in front of their eyes while in operation. Holographic representations are superimposed directly over the patient’s body – no need to look back-and-forth between patient and monitor. Via the headset, needle placements, measurements of incisions, angle and location of screw placements, etc. are all displayed right in front of the doctor’s vision.  

Mixed reality navigation is simply another tool that will become common in hospitals. There was a time when surgeons did not have screens and monitors to guide them during operation. When these were first implemented, surgeons had to learn how to utilize and trust them. Now, those detached 2D displays are being replaced by attached 3D displays. It is simply the next step in the evolution of surgery; the next tool for surgeons to adopt into their trade.   

Is an AR Headset Comfortable Enough for Surgery? 

As incredible an invention as augmented reality surgical navigation is, there are still practical matters like wearability to deal with. As doctors are sometimes in surgery for four or more hours at a time, the overall comfort of a device attached to their head is important.  

Fortunately, AR headsets like Microsoft’s HoloLens 2 were developed for gamers, so comfort was a priority from the very beginning. The HoloLens 2 weighs approximately 1.25 lbs., an excellent weight for prolonged sessions. It also includes a dial-in fit system, which can adjust to your exact head size, making it optimal for extended use. Future models of headsets are sure to be even more ergonomic and comfortable. 

Is the Battery Life of an AR Headset Sufficient for Surgery? 

The battery life on an AR headset can be limited; such is the nature of the lightweight, comfortable structure necessary to provide long sessions of use. You can only fit so much power into such a small, ergonomic device. Microsoft HoloLens, for instance, only gives the user 2-3 hours of active use before it requires charging again.  

That said, there are two ways of extending an AR session for surgery.  

The first option is to plug the headset in using an extension cord. The headset is fully functional when charging, so that is good. Plugging the system in during surgery has its limitations, however, as the surgeon needs to navigate a procedure while they are tethered.  

The second option is to simply purchase a battery power bank, which can hold up to 20,000 mAh. The battery power bank is likely the preferred option, as you can simply place the power bank in your shirt or pant pocket and proceed for many hours with no lapse in use.   

Is Augmented Reality the Future of Surgery? 

With the evolution of surgery and the incredible advancements of technology, there are many innovations developing in our modern age, not the least of which is AR. With groundbreaking developments, massive funding, and FDA approvals, AR surgery is quickly transforming from a futuristic fantasy to a common practice, resulting in capabilities never before known in the field of surgery.  

To learn more about augmented reality surgery and Novarad’s innovative VisAR technology, go to https://www.novarad.net/augmentedreality