Hand tracking palm rotation not working as expected

Unity Development
1

My goal is to create an object that appears in the user's palm (sort of like a floating tablet) when they rotate their hand. To do that, I need to get the rotation of the hand so that I can tell if it's facing the user's headset. The rotation data returned by "handDevice.TryGetFeatureValue(CommonUsages.deviceRotation, out Quaternion handRotation);" seems to completely ignore the rotation of the user's hand. It is however affected by the head rotation for some reason. I did see the warnings about using the rotation data from the gesture system, but wasn't sure if this was related to that, or if i'm going about this the wrong way. Has anyone else attempted something like this?

1 Like
2

@rmuk It was unclear to me whether you need the rotation angles or just need to detect rotation of the palm. If it is the latter case, it would appear that what you might be looking for is GestureInteractionRotation. If this is the case, for the September Software Release the feature is not available at this time. From MLs Unity SDK Release Notes:

MLGestureClassification's GestureTransformRotation and GestureInteractionRotation are not implemented yet and data will not be guaranteed accurate. Currently only the Positions of the Hand Transform and Interaction Point will be recommended to use.

1 Like
3

Yeah I guess just the direction that the palm was facing (really the wrist would work too). I assumed the gesture rotation was related to the rotation of a specific gesture, not the general tracked hand as that seemed to be available via "handDevice.TryGetFeatureValue(CommonUsages.deviceRotation"

I could probably fake it by calculating a triangle between the thumb, pinky and index finger and then using the up vector on that to determine direction (there's probably a simpler/more efficient way, but that's the first thing that popped into my head)

1 Like
4

Hi everyone, yes there is an existing bug that returns the incorrect rotations for the hand and interaction point. We are working on providing the rotations for all of our key points provided by hand tracking. Stay tuned for more details.

For now, see the MRTK Magic Leap Joint provider for reference on how it interprets the joint positions to provide an estimated rotation.

1 Like
5

Thanks for the reply. I had read that note in the docs but wasn't sure if that only applied to the rotation of actual gestures or if it also applied to the rotation/orientation of individual segments like the wrist/center.

1 Like
6

@kbabilinski Hi what is the ETA on the fix for joint rotations, is there a way to just get the hand rotation without reworking the MRTK code? I'm hoping to save time. Thanks

1 Like
7

@stuart_spencer We are hoping to have joint rotations integrated in the November release. Keep an eye out for the release notice on the forums and on our developer portal.

1 Like