Raycast to An UI Object

Give us as much detail as possible regarding the issue you're experiencing.

Unity Editor version: 2022.3.6f1
ML2 OS version:1.4.0-dev1
MLSDK version: 1.4.0-dev1
Host OS: (Windows/MacOS): Windows

I'm trying to do a raycast of the eye gaze to gameobjects in the scene. For 3D objects, the following code works well:

var leftEyePosition = eyes.leftEyePosition;
var leftEyeRotation = eyes.leftEyeRotation;
var leftEyeGazeDirection = leftEyeRotation * Vector3.forward;
var leftEyeRay = new Ray(leftEyePosition, leftEyeGazeDirection);
if (Physics.Raycast(leftEyeRay, out RaycastHit leftEyeRaycastHit, 100))
    // hit something
    var hitPoint = leftEyeRaycastHit.point;
    var _rayLength = leftEyeRaycastHit.distance;
    var gameObject = leftEyeRaycastHit.collider.gameObject;

I could find the gameobject just fine. But when it comes to 2D UI objects, I could not find any game object. I'm using canvas -- see the figure below for my UI structure:
As some posts pointed out, I added a mesh collider to my UI objects and it didn't work. I tried adding mesh renderer and setting Cull Transparent Mesh in Canvas Render to false (not sure if it's helpful) but still cannot raycast to it. I tried to look at the examples but seems that none of them uses Canvas -- Is it bad practice to use Canvas for ML2 development? Is so, what should I do and is there a general guideline for it? If I could still use Canvas, is there a way for me to hit these UIs? This post said something about casting ray to 2D UIs, but I don't think a similar EventSystem exists here.

Any help would be appreciated. Thanks in advance!

The Magic Leap SDK does not include any additional APIs to raycast against Unity UI elements, but you might be able to get more information by testing your scripts logic further. For example, you could try to raycast from the controller or from a specified transform, then test to see if the raycast hits the UI element. I suspect that the raycast does not return the result properly because your UI elements do not having a collider.

In the future, I recommend searching other Unity resources to find answers more quickly, since the issue is not Magic Leap specific: https://discussions.unity.com/

I also recommend the following resources that could make development easier:

MRTK 3 allows you to interact with UI elements using your hands, eyes and controller :Mixed Reality ToolKit 3 Overview | MagicLeap Developer Documentation

Unity Created a learn guide on selecting objects with your eyes using XRI, which can be used to select UI elements if the functionality is extended further:

Thanks for your answer! I did add the collider but it's not working properly. I would experiment with the XR Rig object and see if that solves my problem. Thanks!

Hi, have you solved this problem? I am also have the same error. For normal objects, it works but for ui element, it not.

I have to set the render mode of Canvas to world space and then it works

Hi, are you using 3d colliders or 2d ones. I am used to work with 2d and it doesn't work. But when I change it to 3d, it works.