In a world first, JGH enables Toronto specialist to virtually attend Montreal surgery

Enhanced digital technology streamlines real-time interventional and surgical procedures

The innovative use of advanced digital technology has enabled the JGH to virtually extend its hybrid operating room all the way to Toronto—and there's no limit to how far it can stretch.

This was achieved recently, for what is believed to be the first time in this capacity anywhere in the world, through the enhanced use of Microsoft's HoloLens technology.

In mid-March, a surgical procedure was performed at the JGH, in which the surgeon used a HoloLens2 headset, whose clear lenses covered his eyes like glasses. Whatever he saw through the lenses, including the movement of his own hands, was instantly transmitted to an interventional specialist in Toronto. There the images appeared in real time (and with no significant lag) on an array of computer screens.

As a result, the specialist was able to view all aspects of the procedure with such immediacy and clarity that it was almost as if he was standing in the operating room in Montreal. (Real-time x-ray fluoroscopy, vital sign monitoring, heart ultrasound, and more, were visible on his screen, as a result of work done by Auger Groupe Conseil Inc.)

The procedure itself was performed by the hospital's structural cardiology team, which uses minimally invasive techniques to implant prosthetic devices in the heart.

This means that the JGH, working with Medtronic Canada ULC and Auger Groupe Conseil, now has the capability of obtaining support more quickly and smoothly from specialists and experts—or providing support to them—by virtually taking the hospital to them, no matter where in the world they may be.

Until now, external consultants have had to travel in person to offer advice, supervise or teach about surgical and interventional procedures that are particularly difficult or are not commonly performed.

In some instances, special supervisors (known as “proctors”) also visit to certify the ability of surgeons and interventionalists to perform new procedures.

Better quality of surgical care

For patients, this new development promises better quality of surgical care, with support that can flow in two directions: JGH physicians can turn much more easily to colleagues far beyond Montreal for guidance. Similarly, JGH experts can do the same for those in other centres, which stand to benefit from the JGH's advanced progams in the cardiovascular field and in other medical areas.

This innovation also means that when an external participant needs to be involved, surgery can be scheduled at a time that more closely conforms to the patient's needs, rather than to the travel plans of the visiting expert.

« Everybody who uses this technology is blown away by it,” says Dr. Emmanuel Moss, Chief of Cardiac Surgery at the JGH. “It feels like you really have an expert standing right next to you and helping you do your best for the patient. ».

Dr. Lawrence Rudski, Chief of Cardiology and Director of the Azrieli Heart Centre at the JGH, says the new technology was first used in early March, when a patient underwent a procedure known as a TAVI, a transcatheter aortic valve implantation.

In this minimally invasive procedure, a failing aortic valve is replaced by threading a catheter through the groin or the carotid artery into the heart, and then passing an artificial valve through the catheter.

Before this was done, the three partners conducted pilot projects involving portions of the procedure. Included was the use of three-dimensional augmented reality to guide the preparation of the valve (known as “crimping”). As well, trials were performed with remote specialists in a control room in the hospital or elsewhere in the city, rather than being present in the operating room.

Finally, in March, the end-to-end procedure was performed at a distance. Dr. Rudski, who is Chief Medical Information Officer for CIUSSS West-Central Montreal, says the entire TAVI procedure was monitored by a physician in Toronto, who communicated clearly and easily with the JGH surgeon, who wore a HoloLens2 headset.

Looking to the future

At the JGH, the TAVI is considered a mature program, with outstanding results stemming from collaboration and partnership among physicians, surgeons, nurses and radiology technologists (plus support from many others) in a multidisciplinary team.

However, Dr. Rudski notes, in testing the new technology, everyone approached the operation as if the TAVI was relatively new to the JGH.

The Toronto physician played the role of proctor, a type of supervisor who attends operations to determine whether a particular surgeon has enough training and skill to perform a new procedure on her or his own.

The noteworthy achievement in the March operation was the fact that the “proctor” did not have to be physically present at the JGH—in fact, he was over 500 kilometres away—in order to properly see and thoroughly assess what the surgeon was doing.

Eventually, Dr. Rudski says, the use of the HoloLens technology is expected to substantially reduce the need for some specialists to make time-consuming, tiring and expensive trips from one city to another—or from one hemisphere to another—to attend certain types of operations.

Over the past year, the JGH has piloted the HoloLens for several relatively straightforward activities, such as providing remote guidance in wound care, and troubleshooting certain functions related to the ventilators of COVID-19 patients.

However, with the success of the TAVI procedure, Dr. Rudski says he will be looking into other, more ambitious endeavours. These might include using the HoloLens for electrophysiology (assessing the heart's electrical activity), possibly in partnership with other centres; guiding vascular surgeons in procedures involving the thoracic and abdominal aorta; or providing home care to patients with acute medical conditions.

A head-mounted computer

The focal point of the new technology is a HoloLens headset—in effect, a head-mounted computer—that uses multiple sensors, advanced optics and holographic processing. Clear lenses allow the wearer to view the true-life environment, onto which static images, animation or video can be projected. This is known as “augmented reality.”

The wearer can move the images or advance them by swiping the air where they seem to be hovering. In addition, any image—for instance, a CT scan or the illustration of an anatomical feature—can be virtually superimposed onto the patient's body.

All of these capabilities are already commonplace in the HoloLens experience. What the JGH and its partners have done is to build on the technology by enabling an expert in a remote location to clearly view exactly what the wearer of the HoloLens headset (in this case, the surgeon) is seeing and doing.

The person at the remote location can also receive live digital feeds of any additional information that is needed to perform complex, multi-step interventional procedures. Real images can be mixed with augmented-reality images, using programming that is based on the procedure to be performed.

However, there is no need for a headset to be worn by the remote expert. All that's required is a good internet connection and an array of two or three high-definition monitors to display whatever the surgeon is seeing—actual scenes, images, text, videos, etc.

Not only can the expert and the surgeon speak with each other, the expert can annotate any of the images by writing on an electronic tablet. These notes instantly appear on the lenses of the HoloLens in the surgeon's field of vision, as well as on the expert's screen—all while maintaining three-dimensional positioning.

The end result: Expert and surgeon can interact almost as if they were standing side by side.

“Whenever we've used this system in the operating room, we've never felt we were missing anything by not having that other person physically next to us,” says Dr. Moss. “They can see exactly what I'm doing and I'm able to complete the task seamlessly.”

“Of course, we don't expect to use the HoloLens technology for every procedure,” says Dr. Ali Abualsaud, an Interventional Cardiologist at the JGH. “But if we need to apply our existing skills to a new procedure, the HoloLens makes it possible for an experienced physician to join us relatively easily.

“This is where the HoloLens just shines! The options that suddenly become available to us seem limitless.”

Boosting the creative implementation of digital technology

Dr. Rudski says the notion of using the HoloLens to deliver remote assistance was initially discussed in 2020, at an early point in the COVID-19 pandemic. The sharp cutback in travel resulted in the need for a digital alternative to the in-person participation of a proctor or other visiting specialist.

Medtronic Canada became involved, since it is a leading supplier of products and services in the cardiovascular field, including TAVI valves, coronary stents, pacemakers and defibrillators.

Joining the JGH and Medtronic was Auger Groupe Conseil, a Quebec industrial engineering company that has customized the HoloLens for various purposes, including the complicated, multi-step maintenance of machinery in heavy industry.

Marcel Lafontaine, President of Auger Groupe Conseil, made his company's devices and expertise available to the JGH early in the pandemic, hoping to make the hospital's work environment safer. The subsequent synergy of the teams allowed the JGH and Auger Group to work together with Medtronic on this project.

According to Andrew Topaczewski, Manager of Strategic Partnerships for Medtronic, one of the toughest challenges was eliminating the lag time, so that there would be no discrepancy in what is seen by the surgeon and by the expert at the remote location.

Another significant innovation is that during the TAVI procedure, the images that appear on the screens of the remote expert come through a direct computer feed from the JGH. This produces images of a much higher quality than would be obtained by simply pointing a camera at the JGH screens and relaying those images to the remote location.

The new use of the HoloLens is part of a larger effort—spearheaded by Dr. Lawrence Rosenberg, President and CEO of CIUSSS West-Central Montreal—to boost the creative and innovative implementation of digital technology in health care and social services.

Turning ideas into reality is the mandate of OROT (Hebrew for “lights” or “illumination”), the CIUSSS's innovation hub, which strives to transform the healthcare experience. It does so in ways that are patient-centric and informed by large-scale data science.

Central to OROT's work is the philosophy of “Care Everywhere”—in other words, focusing on the needs of healthcare users and delivering their care wherever they can receive it most easily, most safely and most practically.

“Using the HoloLens for the TAVI procedure fits right into this approach,” says Dr. Rudski. “It's a perfect example of pushing digital technology to a higher level, so that an expert at another location can come to our hospital virtually and participate in surgery, and vice versa.

“This is a great help to our surgical staff, but ultimately, the person who really benefits is the patient.”