Binding hands interaction actions with the controller actions causing issues

OpenXR
1

Hello there,

I'm developing a native openxr application that should support hands and controller interaction, so the user can use the controller by default for interaction if it is switched on, otherwise the user can only use hands for interaction. I'm using "XR_ML_ml2_controller_interaction" extension for the controller interaction and "XR_EXT_hand_interaction" extension for the hand interaction.

I have an issue when I create the hand interaction actions for both hands and the controller interaction actions for the controller considering the handedness at the same time as the following code sample:

// Actions
BaseActionSet = CreateActionSet(1, "base_action_set", "Action Set used on main loop");

OXR(xrStringToPath(Instance, "/user/hand/left", &LeftHandPath));
OXR(xrStringToPath(Instance, "/user/hand/right", &RightHandPath));
XrPath handSubactionPaths[2] = {LeftHandPath, RightHandPath};

// Creating hand interaction actions
HandAimPoseAction = CreateAction(BaseActionSet, XR_ACTION_TYPE_POSE_INPUT, "hand_aim_pose", NULL, 2, handSubactionPaths);
HandGripPoseAction = CreateAction(BaseActionSet, XR_ACTION_TYPE_POSE_INPUT, "hand_grip_pose", NULL, 2, handSubactionPaths);
PinchAction = CreateAction(BaseActionSet, XR_ACTION_TYPE_FLOAT_INPUT, "pinch_value", NULL, 2, handSubactionPaths);

// Creating ml2 controller interaction actions
ControllerAimPoseAction = CreateAction(BaseActionSet, XR_ACTION_TYPE_POSE_INPUT, "controller_aim_pose", NULL, 2, handSubactionPaths);
ControllerGripPoseAction = CreateAction(BaseActionSet, XR_ACTION_TYPE_POSE_INPUT, "controller_grip_pose", NULL, 2, handSubactionPaths);
IndexTriggerAction = CreateAction(BaseActionSet, XR_ACTION_TYPE_FLOAT_INPUT, "index_trigger", NULL, 2, handSubactionPaths);
ButtonMenuAction = CreateAction(BaseActionSet, XR_ACTION_TYPE_BOOLEAN_INPUT, "button_menu", NULL, 2, handSubactionPaths);

Afterwards I call "xrSuggestInteractionProfileBindings" to suggest bindings for all interaction profiles that the app supports (handInteractionProfile and ml2InteractionProfile) as the following:

const std::unordered_map<XrPath, std::vector<XrActionSuggestedBinding>> allSuggestedBindings = GetSuggestedBindings(GetInstance());
  
 
for (auto& [interactionProfilePath, bindings] : allSuggestedBindings) {
    XrInteractionProfileSuggestedBinding suggestedBindings = {};
    suggestedBindings.type = XR_TYPE_INTERACTION_PROFILE_SUGGESTED_BINDING;
    suggestedBindings.next = NULL;
    suggestedBindings.interactionProfile = interactionProfilePath;
    suggestedBindings.suggestedBindings = (const XrActionSuggestedBinding*)bindings.data();
    suggestedBindings.countSuggestedBindings = (uint32_t)bindings.size();
      
    OXR(xrSuggestInteractionProfileBindings(Instance, &suggestedBindings));
}

As an example I will explain on the "HandAimPoseAction" vs "ControllerAimPoseAction" and that applies on the other actions.

In case of having the controller connected, the controller interaction works fine as expected. However when the controller is disconnected, I can not get the Aim Pose for both hands because it seems that the state of the aim pose action is not active. As soon as I comment the line where I create the controller actions, the aim poses of the hands work as expected and the hand interaction in general works as expected.

I'm wondering if there is something wrong with my action bindings or if you have any known issue that might be causing that issue.

could you please guide me to fix this issue ?

ML OS version: 1.4.0-dev1

Thank you

3

Hi @m.soliman
I am looking into this issue and I will get back to you as soon as I have some advice. Thank you for the code snippets that will help when I bring this to other engineers.
-Sidney

4

Hi @m.soliman
There should be a fix coming in the next release that should make this work as expected. I hope that helps.
-Sidney

1 Like
5

I've upgraded the OS to v1.4.0-dev2 and that fixed this issue.

Thank You

1 Like