# XRI Hand controller vs UI **URL:** \ **Category:** Unity Development\ **Tags:** hand-tracking, input, interactions\ **Created:** [January 16, 2024, 3:41pm UTC](https://forum.magicleap.cloud/t/xri-hand-controller-vs-ui/3035 "2024-01-16T15:41:54Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![maks](https://avatars.discourse-cdn.com/v4/letter/m/96bed5/32.png) [@maks](https://forum.magicleap.cloud/u/maks)\ **Post date:** [January 16, 2024, 3:41pm UTC](https://forum.magicleap.cloud/t/xri-hand-controller-vs-ui/3035/1 "2024-01-16T15:41:54Z")
[![](https://us1.discourse-cdn.com/flex019/uploads/magicleap1/original/2X/0/0f353bb5d9fa4b7a7d127ad45cd5651f3dd2db9f.jpeg "Magic Leap XRIHand controller vs UI") ](https://www.youtube.com/watch?v=EfZyAVROp4s) Hey everyone Been struggling with this for a while — ML Controller works perfect with it's Ray Interactor and both 3D and UI elements. Where hand controls provided by XRIHand controller do not interact with UI objects but do interact with other XR Intractables. Any fix available? ---
**Author:** ![etucker](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/etucker/32/842_2.png) [@etucker](https://forum.magicleap.cloud/u/etucker)\ **Post date:** [January 16, 2024, 10:38pm UTC](https://forum.magicleap.cloud/t/xri-hand-controller-vs-ui/3035/2 "2024-01-16T22:38:33Z")
Hi @maks, Thank you for reaching out regarding this issue. I have consulted the team and I will report back as soon as I learn more. Could you please provide a few extra details, though? - Unity Editor version - ML2 OS Version - ML SDK Version - Development OS (e.g. Mac, Windows) ---
**Author:** ![maks](https://avatars.discourse-cdn.com/v4/letter/m/96bed5/32.png) [@maks](https://forum.magicleap.cloud/u/maks)\ **Post date:** [January 17, 2024, 8:29am UTC](https://forum.magicleap.cloud/t/xri-hand-controller-vs-ui/3035/3 "2024-01-17T08:29:45Z")
Hi @etucker 2022.3.10f1 1.4.1 1.4.0 MacOS ---
**Author:** ![kbabilinski](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/kbabilinski/32/19_2.png) [@kbabilinski](https://forum.magicleap.cloud/u/kbabilinski)\ **Post date:** [January 17, 2024, 3:48pm UTC](https://forum.magicleap.cloud/t/xri-hand-controller-vs-ui/3035/4 "2024-01-17T15:48:00Z")
@maks do you mind providing more information on how you are mapping the hand tracking input to the XRI controller? ---
**Author:** ![maks](https://avatars.discourse-cdn.com/v4/letter/m/96bed5/32.png) [@maks](https://forum.magicleap.cloud/u/maks)\ **Post date:** [January 17, 2024, 3:51pm UTC](https://forum.magicleap.cloud/t/xri-hand-controller-vs-ui/3035/5 "2024-01-17T15:51:50Z")
@kbabilinski I followed this tutorial > **[Hand tracking and gestures - Unity Learn](https://learn.unity.com/tutorial/hand-tracking-and-gestures?uv=2022.2&courseId=643419cbedbc2a5f42275126)** > > Hand tracking is a feature that allows users to interact with applications without the need for controllers or other input devices. The Magic Leap 2 uses a combination of cameras and sensors to detect and track hand movements in real time, which... I saw you answering once on the same topic with advice to use MRTK instead, but the XRI solution works so fine except this UI issue. Can we may be find a solution that will make it work? ---
**Author:** ![kbabilinski](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/kbabilinski/32/19_2.png) [@kbabilinski](https://forum.magicleap.cloud/u/kbabilinski)\ **Post date:** [January 17, 2024, 7:29pm UTC](https://forum.magicleap.cloud/t/xri-hand-controller-vs-ui/3035/6 "2024-01-17T19:29:30Z")
It's a little hard to tell, does the UI panel react to the highlight input events from the ray when it hovers over the buttons? If it works, you may need to edit the XRI Hand Controller script to send the additional UI Input events by editing the following section of the code ```csharp bool isPinched = GetPinch(); controllerState.selectInteractionState.active = isPinched; controllerState.selectInteractionState.activatedThisFrame = isPinched && !pinchedLastFrame; controllerState.selectInteractionState.deactivatedThisFrame = !isPinched && pinchedLastFrame; ``` Alternatively , You can use the latest version of the Magic Leap Unity SDK 2.0 and OpenXR to take advantage of the XR Hands and XRI samples. [Hands Interaction Demo | XR Interaction Toolkit | 2.5.2](https://docs.unity3d.com/Packages/com.unity.xr.interaction.toolkit@2.5/manual/samples-hands-interaction-demo.html) Note, to make the sample work properly on Magic Leap 2, you will need to make sure the Unity Input Asset is updates with the correct bindings. By default the hand input maps to the MetaHand input profile and has to be changed to the HandInteraction paths. ---
**Author:** ![maks](https://avatars.discourse-cdn.com/v4/letter/m/96bed5/32.png) [@maks](https://forum.magicleap.cloud/u/maks)\ **Post date:** [February 15, 2024, 4:16pm UTC](https://forum.magicleap.cloud/t/xri-hand-controller-vs-ui/3035/7 "2024-02-15T16:16:10Z")
Hey @SmiledPenguin47 No it does not. But the ray is reacting — it "sees" that it's hovered over something it can interact with. the section of the code you've mentioned looks identical to what you've suggested bool isPinched = GetPinch(); ``` controllerState.selectInteractionState.active = isPinched; controllerState.selectInteractionState.activatedThisFrame = isPinched && !pinchedLastFrame; controllerState.selectInteractionState.deactivatedThisFrame = !isPinched && pinchedLastFrame; pinchedLastFrame = isPinched; ``` so what should I add? ---
**Author:** ![kbabilinski](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/kbabilinski/32/19_2.png) [@kbabilinski](https://forum.magicleap.cloud/u/kbabilinski)\ **Post date:** [February 15, 2024, 6:13pm UTC](https://forum.magicleap.cloud/t/xri-hand-controller-vs-ui/3035/8 "2024-02-15T18:13:40Z")
We will look into this but you may get an answer faster by asking this question on the Unity Forum or Unity Community since they support XRI directly. In the meantime, I recommend trying our MRTK3 which supports hand interaction across 2D and 3D objects. Including poke interaction. ---
**Author:** ![maks](https://avatars.discourse-cdn.com/v4/letter/m/96bed5/32.png) [@maks](https://forum.magicleap.cloud/u/maks)\ **Post date:** [February 16, 2024, 8:43am UTC](https://forum.magicleap.cloud/t/xri-hand-controller-vs-ui/3035/9 "2024-02-16T08:43:16Z")
Actually it started to work after I added this line controllerState.uiPressInteractionState.active = isPinched; ---
**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/magicleap1/original/1X/bd9b737a4d45ea243aa7eb177d60a773b806899a.svg) [@system](https://forum.magicleap.cloud/u/system)\ **Post date:** [March 2, 2024, 8:44am UTC](https://forum.magicleap.cloud/t/xri-hand-controller-vs-ui/3035/11 "2024-03-02T08:44:08Z")
This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.