Bug in "Disable Controller When Not In Hand" logic in ML MRTK Package

Unity Development
,
1

ML MRTK Package 1.8.0

We noticed some annoying behaviour with the controller disabling when the hand tracking is lost. After some digging, it looks like a bug in StateToHandedness() inside MLControllerHandedness.

This function attempts to tell MRTK which hand the controller is in. Most of the time the lower-level input system can determine which hand the controller is in and that is passed straight through to the MRTK. However, when hand tracking is lost the lower-level input system just returns MLInputControllerHand.None. At this point StateToHandedness() attempts to make a guess which hand the controller is in. Sensibly it attempts to return the previously assigned handedness unless the controller is more than a set distance away from the head.

This distance from head check is the problem. The logic is comparing the controller position returned from the input system with the world space camera position. However, the controller position is returned in a different coordinate frame to the camera. This logic test then fails because it thinks the controller is very far from the head and then turns off the controller.

All other uses of the controller position first transform the controller position by the MixedRealityPlayspace to put it in the right coordinate frame

MixedRealityPlayspace.TransformPoint(controllerPosition);

If you change the logic in StateToHandedness to do this too, then the controller will remain on and in the correct hand even when the hand tracking is lost. We've made this change locally and the behaviour is much nicer.

It might be worth reviewing the whole of that function. Some of the logic seems to be redundant which makes it harder to follow.

3

Hi @edd,

Thank you so much for taking your time to bring this to our attention. I will go ahead and log this issue and report this information to our team as soon as possible. Does your workaround suffice for now or are you currently blocked on something?

Best,

El

4

Yes, the workaround is working well, so we're not blocked.

When you pass on the info to your team, it's probably worth mentioning the easy way to reproduce this is to bring the controller up to your face so the headset can't see your hands anymore. In your implementation, the controller will disappear, with the fix-ups the controller remains active.

5

@edd Appreciate the heads up, it looks like a solid way to address this and we will look into making this improvement in the future comes standard with the package.

2 Likes