NA-MIC Project Weeks

Back to Projects List

Collaborative AR application using OpenIGTLink, 3D Slicer and Magic Leap 2

Key Investigators

Alicia Pose Díez de la Lastra (Universidad Carlos III de Madrid, Madrid, Spain)
Javier Pascau (Universidad Carlos III de Madrid, Madrid, Spain)
Steve Pieper (Isomics, USA)
Ron Kikinis (BWH, USA)

Project Description

In previous Project Weeks, we showed a way to communicate Microsoft HoloLens 2 with 3D Slicer using OpenIGTLink protocol. The current solution is implemented in a 3 elements system. It is composed by A Microsoft HoloLens 2 headset, the Unity software, and the 3D Slicer platform. The HoloLens 2 application is not directly built on the device, but remotely transferred from Unity in real time using Holographic Remoting. The communication workflow is represented in the following image:

The results of that work are publicly available at this GitHub repository.

In more recent Project Weeks we evaluated the transferability of the aforementioned project to other AR devices. Specifically, we focused on the VARJO XR-3 headset, obtaining fast and high-quality results:

This time, we aim to follow a similar approach, this time with Magic Leap 2, with the ultimate goal of creating a colaborative application between multiple headsets communicating through OpenIGTLink.

Objective

Objective A: Adapt previous Unity apps to Magic Leap 2.
Objective B: Exchange information between Magic Leap 2 and 3D Slicer.

Approach and Plan

Adapt previous Unity apps to Magic Leap 2.
Test Holographic Remoting in Magic Leap 2.
Test the exchange of information between Magic Leap 2 and 3D Slicer.
Add a new headset to the system and try to exchange information between HoloLens 2 and Magic Leap 2

Progress and Next Steps

Bring both a HoloLens 2 and a Magic Leap 2 headsets to PW.
Also tested with Viture Luma Ultra with Neckband - confirmed that Unity SDK can be used to develop Viture apps and should be able to use core code as Magic Leap 2
- Viture native SDK released on Thursday