The technology allows clinicians and patients to view and interact with high-resolution 3D hologram-like images of patient scans, alongside traditional imaging techniques, and is being used to support some patient consultations, assist preparations for certain operations, and provide immersive clinical training. The Trust explains that ‘mixed reality, which blends 3D digital content with the real world’, is regarded as ‘the next frontier’ in visual computing. It enables users to interact with digital content – such as 3D renderings of patient scans – while remaining fully aware of their real-world surroundings. “Unlike Virtual Reality (VR), which immerses users in a completely separate 3D environment and blocks out the real world,” the Trust explained, “mixed reality enhances the user’s perception by integrating digital and physical elements.” The Trust’s neurosurgery team utilizes XARlabs’ simXAR mixed reality tool, which was developed independently by a registrar at the Trust when he was part of the NHS Clinical Entrepreneur Programme, prior to joining the Trust.
The simXAR technology was initially planned to support training within the neurosurgery department with funding from NHS England. The Trust neurosurgery team recognised the broader benefits the technology could offer patient care, and introduced it in some consultations to better explain proposed treatments to patients as part of the consent process, as well as using it more widely to enhance team preparedness.
The team uses the technology to convert CT and MRI patient scans into 3D augmented reality models. During training sessions and certain patient consultations, both the surgeon and the trainee, or the surgeon and the patient, wear Trust-owned Magic Leap 2 headsets. Patients provide consent for both the generation of their models, and for wearing the headsets. The surgeon can interact with the 3D model by zooming in, rotating it, or even ‘walking’ inside to visualise the patient’s anatomy ‘in an immersive and easily understandable way’. This marks a significant shift from the traditional method of viewing scans one series at a time on 2D computer screens. The team believes this represents an innovative way of viewing medical imaging that can significantly benefit both patients and health professionals in the NHS.
The Imperial College Healthcare neurosurgery team is primarily using the simXAR technology in training and simulation sessions, and has also begun inviting some patients to view their 3D models before surgery to help them better understand the planned procedures. The technology is also being used alongside conventional scans to enhance the team’s ability to view and understand the surgical anatomy ahead of certain brain and spine operations. Additionally, team members are looking to assess the technology’s potential broader benefits in medical training, patient experience, and surgical guidance, subject to ethical and regulatory approvals.
Arthur Dalton, Consultant skull base and vascular neurosurgeon, said: “We immediately saw the potential of this technology, and set out to develop a world-class mixed reality suite, drawing on the expertise and innovation within our own surgical team. The level of detail and interactivity it offers far surpasses traditional methods, facilitating better preparation and understanding for surgeons, trainees and patients.
“We’re already starting to see benefits as we continue embedding this system into our work. The potential for improving patient care and training is huge. We believe we are the first, or one of the first, to adopt this type of mixed reality platform in this way in the NHS, and hope our work can help the NHS remain at the forefront of this emerging technology.”