Hands do not work after restarting OpenXR Loader with MRTK3

#Description

In my MagicLeap 2 project, at one point I stop and deinitialize the active OpenXR Loader and then some time later I re-initialize and start it. After reinitialization I cannot see hands visualized and interactions with hands do not work.

To reproduce

Run the application with default MRTK3 Rig or MRTK3 MagicLeap Rig and a basic UI with slate prefab and buttons. At one point ( for example with a button click) stop and deinitialize active OpenXR Loader. Then after some time (10-15 seconds) re-initialize and start the active loader. Now you can cannot see your hands visualization and cannot interact with anything via hands but you can use controller normally.

  • Unity Version : 2022.3.15f1
  • MRTK Version : v3.0
  • SDK Version : 2.0.0
  • MRTK3 magicLeap Integration Package : 1.0.0-pre6

@a.nourinia Thank you for this information. To make sure I understand correctly, are you enabling Magic Leap as the XR Plugin Provider or are you enabling Magic Leap Support via OpenXR ? .

Note, at this time our MRTK 3 and 2.8 Magic Leap Packages do not support the OpenXR implementation of our SDK. We recommend that you continue to use the Magic Leap XR Plugin Provider instead of OpenXR.

Thank you for your reply.

I am using Magic Leap as the XR Plugin Provider.

Thank you, I have reported this issue to the engineering team.

Do you mind providing some background about this functionality? What are you trying to achieve and are you able to work around the problem? Have you noticed this behavior in the previous version?

Apart from Unity's OpenXR instance, we have our own instance that we use for our purposes. Then, we want to go back to Unity application and keep using it.

Yes we tried the two previous MRTK3 Magic Leap packages, it was also not working

I'm sorry, I don't understand, can you elaborate on this some more?

Apart from Unity's OpenXR instance, we have our own instance that we use for our purposes. Then, we want to go back to Unity application and keep using it.

Does the hand tracking stop working after going into another application and then resuming it?

We have two parts of our app and each of them creates its own OpenXR Instance and Session with their own features. We start with a simple Unity scene and at one point we destroy instance Unity uses and create and use our OpenXR instance and session. After we are done with our part we go back to Unity, and resume the scene after re-initializing the Instance and Session via XR.Management package. The problem occurs when we go back to Unity scene and start OpenXR Loader after we are done with our own OpenXR session and instance. Hand tracking does not work. We tried same thing with the MagicLeap2 MRTK 3 fork/sample, the problem also happens in the sample project. So, in the sample project, you can Stop and Deinitialize OpenXR activeLoader (via a button click for example) and after some time re-initialize and start it. After you restart is you cannot use or see your hands but controller works fine.

To make sure I understand, your application switches between the Magic Leap XR Subsystem and the OpenXR subsystems. I assume you are using something like

   XRGeneralSettings.Instance.Manager.StopSubsystems

Do you mind sharing the use case on disabling and enabling the two separate systems?

Are you switching between the Magic Leap OpenXR Provider and the Magic Leap XR Plugin Provider? Do both of these scenes use MRTK 3?

We have one scene (with MRTK 3) and we do not change the provider, we only use Magic Leap provider. While stopping and restarting , yes , we use the following codes :

XRGeneralSettings.Instance.Manager.activeLoader.Start()
XRGeneralSettings.Instance.Manager.activeLoader.Initialize();

XRGeneralSettings.Instance.Manager.activeLoader.Stop()
XRGeneralSettings.Instance.Manager.activeLoader.Deinitialize();

Also we tested this behaviour with Magic Leap MRTK 3 for sample. It also does not work there. But if we apply the same logic for the original MRTK 3 sample repository with HoloLens 2, it works fine on that sample.

Thank you for clarifying. Do you mind providing additional information as to why you are stopping and restarting the provider?

A part of our application depends on our custom actions and instance that's why we stop Unity's OpenXR instance and use ours for some time. And then we restart Unity's OpenXR instance.

@a.nourinia I reported this bug to our engineering team and will let you know when I have any additional information.

Can you verify that you are using the following sequence to shutdown and restart the loader. The proper order should be:

 Initialize(), Start(), Stop(), Shutdown(), Initialize(), Start()

I'm working with @a.nourinia
Yes, we are using that order to shutdown and restart the loader and actually the function that we are using, is called Deinitalize not Shutdown.

//  The default initialization when the app starts
XRGeneralSettings.Instance.Manager.activeLoader.Initialize();
XRGeneralSettings.Instance.Manager.activeLoader.Start();

// When we move to the second part of our application we stop and destory 
// the current openxr instance and session to create new ones in the second part.
XRGeneralSettings.Instance.Manager.activeLoader.Stop();
XRGeneralSettings.Instance.Manager.activeLoader.Deinitialize();

// When we go back to the first part of our application and 
// try to recreate the openxr instance and session, we can't the hands working again.
XRGeneralSettings.Instance.Manager.activeLoader.Initialize();
XRGeneralSettings.Instance.Manager.activeLoader.Start();

Thank you for that information. We are looking into this issue.

After shutting down they system please try explicitly restarting the hand tracking and eye tracking (if needed)

UnityEngine.XR.MagicLeap.InputSubsystem.Extensions.MLHandTracking.StartTracking();

Thank you this solved the issue.

2 Likes

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.