Issue with Raycasting on Trackable Planes in Magic Leap 2

OpenXR
1

Unity Editor version 2022.3.47f1:
ML2 OS version: 1.12.0
MLSDK version: 2.6.0

Hi everyone,

I am developing an application for Magic Leap 2 where I scan the environment for planes using the sample scene provided. Once the plane detection is complete, I stop the scanning process and attempt to use raycasting to interact with the detected plane.

However, I am facing some issues:

  • When using the controller pointer, it passes through the detected plane, making interaction difficult.
  • If I enable a mesh, the pointer does not pass through, but I am unable to instantiate a sphere on the detected plane or mesh.

I would like guidance on how to correctly use raycasting to detect trackable planes and instantiate objects at the intersection point.

Here are my current SDK versions:

ML2SDK: 2.6.0
OpenXR: 1.13.0
AR Foundation: 5.1.6

Any insights or suggestions would be greatly appreciated!

Thanks in advance.

2

You may need to change the layer that the panes are on. You may also need to add a mesh collider to the plane.

Unfortunately, it’s hard to tell directly what might be happening. But we do have this guide that might be helpful.

3

Hello,

I tried your approach, but it doesn’t seem to work. Let me explain my issue simply.

I am performing plane detection using the Magic Leap 2 sample project, and I can successfully see the detected planes. Now, I want to draw a rectangle on the detected plane using the controller. My approach is to cast four spheres on the plane and then use a Line Renderer to connect the points.

However, I am facing an issue with Raycasting in Magic Leap 2:

  1. I am unsure if I am correctly casting the ray using the trigger button.
  2. Even if the ray is being cast, the detected plane does not seem to recognize it.

I need your help to understand and resolve these issues. Could you guide me on how to properly perform raycasting on detected planes using the controller?

Looking forward to your response.

4

Does the plane have a mesh collider attached to it? Are you simply using physics raycast in Unity? Have you added logs ? (For example log when the raycast hit’s a collider)

5

Yes, the plane does have mesh collider attach to it. I make no changes to the sample from magic leap2 SDK. but still the pointer ray passes through the plane. Also I tried both methods, AR raycast and Physics Raycast, non of them works.

6

Hmm, it sounds like a tricky problem. Can you verify that you get the results you are expecting when using the physics raycast on a cube or other 3d mesh?

7

Hello,

I have resolved the issue. I set up a new project and added the AR Raycast Manager, along with debug logs for troubleshooting. As a result, I can now see the detected plane.

Using the trigger button, I instantiate a sphere on the plane. However, the sphere does not align with the controller’s pointer; instead, it always spawns with an offset.

Below is the simple logic I used for raycasting:

void Update()
{
// Get the controller pointer position & rotation
Vector3 pointerPos = pointerPosition.action.ReadValue();
Quaternion pointerRot = pointerRotation.action.ReadValue();

    Ray ray = new Ray(pointerPos, pointerRot * Vector3.forward);

    // AR raycast
    if (arRaycastManager.Raycast(ray, arHits, TrackableType.PlaneWithinPolygon))
    {
        Pose hitPose = arHits[0].pose;

        // (Trigger Pressed)
        if (triggerPressed.action.WasPressedThisFrame() && !shapeClosed)
        {
            PlacePoint(hitPose.position);
        }
    }
}

Could you help me understand why this offset occurs and how to properly align the sphere with the controller’s pointer?