Hands are rotated incorrectly with MRTK + OpenXR

As part of updating to a newer MLSDK release, we have switched to using OpenXR rather than the native ML MRTK implementation. Everything is working, however the hands are rotated 180 degrees and 90 degrees across two different axes. This is the case for both left and right hands. I have seen that the unity sample for 2.4.0 works correctly but it does not use the MRTK integration. Since he sample in the mrtk fork also is set up for MagicLeap vs OpenXR, I can't just use it to figure out configurations and when I tried to set that one up for OpenXR, the hands just don't work. Not sure if there is something obvious I am missing.

Unity Editor version: 2022.3.13
ML2 OS version:1.9.0
MLSDK version: 2.4.0
MRTK version: 3.0.0

Error messages from logs (syntax-highlighting is supported via Markdown):
n/a

Configuration pics:





image
(using a customized Head Gaze prefab that we have set up to switch on and off depending on if hands are in the scene, for gaze and dwell functionality)

I noticed Runtime Config Enabled is not checked:

image

and there's an OpenXR version of the MRTKProfile that you can assign here:

2 Likes

Hey @peter.nolen

In addition to what @martin.wilter said, there are a few more things you could try.

Currently, there are known issues with the MLSDK 2.4.0 and specific Unity versions, all relating to controller/hand tracking. These seem to be related to eye tracking changes made in that version of the SDK. Investigation and solutions are still in progress by our engineers.

Option 1:
You could try to disable everything related to eye tracking.

  1. In the OpenXR Features, disable all eye tracking support for ML2.
  2. Disable all eye tracking permissions
  3. Remove eye tracking support from any interaction profiles you are using, including any MRTK profiles.

Option 2:
Try downgrading your MLSDK version to 2.3.0 or earlier and see if that alleviates any issues you are experiencing.

1 Like

Ok I removed all eye tracking permissions, I switched to the MRTKProfile-MagicLeap in the package, and I downgraded the MLSDK to 2.3.0 and the Magic Leap MRTK3 to 1.1.0. It's still having the same isuse.

When you say it's with specific unity versions, should I just try updating to the newest LTS for 2022? I know the one we're using is kind of old (you know the old adage, never update your unity version).

You could try out Unity version 2022.3.42 to see if that works.

Otherwise, I think it might be a settings issue or something similar.

But before we dig into settings more, could you try downloading and running the Unity Examples project from the Magic Leap Hub?

Here's a guide to help you through the process.

Please test out the Hands example on your device and see if the hands are aligned properly or not.

If that example works as expected, that would point to some project specific issue going on which we can then try to pin down for you!

@peter.nolen

You can reference the mrtk3_MagicLeap2_OpenXR branch of the public Magic Leap MRTK3 repo to see the OpenXR specific configuration for an MRTK3 project on the ML2. We provide multiple branches to target specific configurations for reference:

One issue I think I noticed with your config is that when using OpenXR, you will need to include Unity's XR Hands package. I didn't see the XR Hands subsystem show in the MRTK3 profile, which seems to imply that package isn't in the project, but the below is what your profile settings should look like (using our provided OpenXR profile):

Since you appear to be using the OpenXR rig variant we provide, you don't necessarily need to enable Runtime Rig Config - though it doesn't hurt anything to enable it even when using the rig variants we provide. But if you ever wanted to use the default MRTK3 rig for cross-platform development, definitely enable that so the default rig will get configured properly.

As mentioned, there might be an issue enabling the experimental Eye Tracker feature with SDK 2.4.0 and getting controller tracking. MRTK3 utilizes a different mechanism for eye tracking than that feature with SDK 2.4.0, so you don't necessarily need it to get eye tracking to work with MRTK3. The OpenXR branch should show the configuration needed to have that all working.

2 Likes

It didn't show the XR Hands subsystem because at the time I didn't have the ml mrtk3 fork and I was using the standard MRTK3.0.

I got that sample project and built it and it worked even in that unity version (I had tried the mrtk3 fork sample but didn't realize there were different branches of it). I then went back and fought with mine again, eventually getting it to pretty much match exactly and was still having issues and I don't know how I stumbled upon it but I did realize that the MLInput had different binding properties for the LeftHand > Position even though it was in the package and the package was correct.

So I did a Reimport All.

Apparently that was my issue the entire time. :neutral_face:

Thanks for all the assistance on this. Sorry it ended up being something so silly. I guess the Unity Library got tripped up when I switched the version of the MLSDK or the ML MRTK3 package. Oof.

1 Like

No need to apologize!
Glad you were able to get it worked out.

We're always happy to help in any way we can!

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.