Magic Leap 2 for moving Platform / in zero gravity - modifying head pose tracking

General
,
1

Hello,
As part of the XR Lab at the European Space Agency (ESA), we are currently developing an AR application for the use on the ISS. The Magic Leap 2 is particularly useful for us because we can easily access the video and depth cameras. Previous tests with various VR headsets have shown that it is quite challenging to use them in space. In microgravity, the readings from IMUs and magnetometers do not behave the same as they do on Earth. Most XR headsets do not know how to interpret this data and lose tracking.

As a first test for the Magic Leap 2, we tried using it in a car to see what happens when the IMU readings conflict with the camera data. As expected, the headset lost tracking.

We are reaching out to find ways to modify the head pose tracking of the headset. By correcting or disabling IMU readings for the head pose tracking algorithm, it should be possible to get a sufficient result for a moving platform or other extreme conditions. We are aware that the Magic Leap does not support a travel mode, but this is essentially what we need. We could implement this mode ourselves, for example by implementing our own SLAM algorithm in the application. However, when the Magic Leap loses tracking, it shows a warning that blocks all other applications. This makes us suspect that such a tracking algorithm would need to be implemented at the driver or firmware level.

We are aware that this problem is quite niche and complicated. We are willing to implement such a solution ourselves, but we need access to the head pose tracking algorithm of the device (signing an NDA for this is not a problem). Alternatively, we could also provide funding if you are willing to develop a customized solution for us.

Any help on this matter would be highly appreciated.

2

Unfortunately, we don’t provide access to disable the IMU sensors or modify the head pose tracking on the Magic Leap 2. The device relies heavily on full 6DoF tracking, and a lot of its functionality is built around that assumption. So the ability to adjust or tweak the head pose logic isn’t something we support.

That said, I’ll be sure to pass along your feedback to our customer care team as a feature request. While it’s not currently on our roadmap, it is interesting to hear how people are pushing the device.

3

This is unfortunate, it makes the MagicLeap unusable for our use case. It would already help if we could turn off the “Tracking, hold still” warning and use the MagicLeap as a kind of static device, simply displaying an image regardless of the current device pose. This way, we could do all the pose estimation in the Unity application and implement our own algorithm.

You don’t support such a mode, right? I have seen a few posts on the forums discussing the need to disable this warning (How to use in low light environment - #2 by kbabilinski and How to disable the tracking lost warning for example). I really think this would be a helpful feature for many use cases (e.g. a static HUD).

4

You are correct. We do not support disabling head pose tracking or disabling the “tracking, hold still message”