OpenXR Marker Understanding Feature finding markers, but returning a zeroed out pose

@chaseryals219 and @tyler.morris
I finally managed to get some build env issues on my end squared away and I got the above script into our examples project with some added debugging and I am getting values for the marker. I also tested with the markertracking example scene and was getting correct behaviour. So maybe not a bug on our part, but probably something easy to mistake in setup. Lets try to figure this out.

-Sidney

This is my modified script

using System;
using System.Collections;
using MagicLeap.Android;
using Unity.XR.CoreUtils;
using UnityEngine;
using UnityEngine.XR.MagicLeap;
using UnityEngine.XR.Management;
using UnityEngine.XR.OpenXR;
using UnityEngine.XR.OpenXR.Features.MagicLeapSupport;
using static UnityEngine.XR.OpenXR.Features.MagicLeapSupport.MagicLeapMarkerUnderstandingFeature;

public class OpenXR_MLMarkerTracker : MonoBehaviour
{
    [SerializeField] private MarkerType type;
    [SerializeField] private int markerNumber;
    [SerializeField] private MarkerDetectorProfile profile;
    [SerializeField] private float markerSize;
    [SerializeField] private ArucoType arucoType;
    
    private MarkerDetectorSettings _detectorSettings;
    private XROrigin _xrOrigin;
    
    public MagicLeapMarkerUnderstandingFeature MarkerFeature { get; private set; }

    public event Action<Pose> MarkerFound;

    private IEnumerator Start()
    {
        Debug.LogWarning("------------------ START script --------------------");
        Permissions.RequestPermissions(new []{MLPermission.MarkerTracking,MLPermission.SpatialAnchors}, response=>{print($"{response}: permissions granted");}, _=>{Debug.LogError("permission denied");}, _=>{Debug.LogError("permission denied");});

        XRManagerSettings manager = XRGeneralSettings.Instance.Manager;
        yield return new WaitUntil(() => manager.isInitializationComplete &&
                                         manager.activeLoader);
        Debug.LogWarning("------------------ Done waiting --------------------");
        if (manager.activeLoader.GetType() != typeof(OpenXRLoader))
        {
            Debug.LogError("OpenXRLoader is not the current XR loader. Disabling script.");
            enabled = false;
            yield break;
        }

        MarkerFeature = OpenXRSettings.Instance.GetFeature<MagicLeapMarkerUnderstandingFeature>();
        if (!MarkerFeature)
        {
            Debug.LogError("Could not find MagicLeapMarkerUnderstandingFeature. Disabling script.");
            enabled = false;
            yield break;
        }

        Debug.LogWarning("------------------ Found feature --------------------");
        switch (type)
        {
            case MarkerType.Aruco:
                SetupArucoSettings();
                break;
            case MarkerType.AprilTag:
            case MarkerType.QR:
            case MarkerType.EAN13:
            case MarkerType.UPCA:
            case MarkerType.Code128:
            default:
                break;
        }
        Debug.LogWarning("------------------ Creating marker detector --------------------");
        MarkerDetector markerDetector = MarkerFeature.CreateMarkerDetector(_detectorSettings);
        if (markerDetector == null)
        {
            Debug.LogError("Could not create a marker detector. Disabling script.");
            enabled = false;
            yield break;
        }

        Debug.LogWarning("------------------ detector created --------------------");
        _xrOrigin = FindAnyObjectByType<XROrigin>();
        if (_xrOrigin == null)
        {
            Debug.LogError("There is no XR Origin in the scene. Disabling script.");
            enabled = false;
        }
    }
    
    private void SetupArucoSettings()
    {
        Debug.LogWarning("------------------ setting aruco settings --------------------");
        _detectorSettings.MarkerType = MarkerType.Aruco;
        _detectorSettings.MarkerDetectorProfile = profile;
        _detectorSettings.ArucoSettings.ArucoType = arucoType;
        _detectorSettings.ArucoSettings.EstimateArucoLength = true;
        _detectorSettings.ArucoSettings.ArucoLength = markerSize;

        _detectorSettings.AprilTagSettings.EstimateAprilTagLength = false;
        _detectorSettings.QRSettings.EstimateQRLength = false;
    }
    
    public void Update()
    {
        Debug.LogWarning("------------------ updating marker detectors --------------------");
        MarkerFeature.UpdateMarkerDetectors();
        foreach (var markerDetector in MarkerFeature.MarkerDetectors)
        {
            if (markerDetector.Status != MarkerDetectorStatus.Ready) 
                continue;
                
            if (markerDetector.Data.Count <= 0) 
                continue;
                    
            foreach (var marker in markerDetector.Data)
            {
                print($"Found marker {marker.MarkerNumber} at pos {marker.MarkerPose.Value.position}");
                Debug.Log("Marker found.");
                if (marker.MarkerNumber != (ulong)markerNumber)
                    continue; 
                    
                Debug.Log("Marker ID matches");
                    
                //This is always zeroed out
                Pose? markerPose = marker.MarkerPose;
                if(markerPose is null)
                    continue;
                    
                Debug.Log($"Marker Position: {markerPose.Value.position.ToString()}, Marker Rotation: {markerPose.Value.rotation.ToString()}");
                Transform originTransform = _xrOrigin.CameraFloorOffsetObject.transform;
                Pose offsetPose = new Pose(
                    originTransform.TransformPoint(markerPose.Value.position),
                    originTransform.rotation * markerPose.Value.rotation
                );
                MarkerFound?.Invoke(offsetPose);
                return;
            }
        }
    }
}

some of the logs from my most recent test using that script


and the hierarchy and settings

We tried adding in your modifications and enabling the Spatial Anchors Subsystem in the OpenXR provider and we're still having the same issue. Is it possible there's a conflict with using MRTK3 and OpenXR?

I will try adding MRTK3 to the test project I have here and I will see if it still works.

If you're needing any screenshots of my project set up or OpenXR & MRTK settings, let me know

I tried the 2.2 SDK example apk and the marker tracking was working there. Unfortunately, since it's an apk, I can't compare the project configuration to ours.

Here are our current OpenXR settings:

Magic Leap Permissions:
image

MRTK3 Magic Leap Settings:

Ok, I had a chance to add some mrtk3 ui just to see if that was the conflict and it is still working. I am going to post all the even possibly relevant things I can find. My other thought is maybe its unity version? I am using 2022.3.30f1, I am not sure what @aheaney was using. Because this is a sample project I do have pretty much all the features and permissions turned on :woman_shrugging: ?




I'll match these settings here in a bit and test them on my end.

I'm using Unity Version 2022.3.16f1 in case we determine that to matter.

Can you post your MRTK Magic Leap settings please @sfernandez?

Sorry missed that one


I tried enabling all of these settings, but the issue is still happening on our end.

I have done the same and am also still having issues

I'll try creating a blank project with just the marker tracking to see if I can reproduce it there.

So I made a new project and copied the Magic Leap and OpenXR settings without adding MRTK and the issue is still happening for me, so I don't think the issue is MRTK related.

This is also using the HandsDemoScene from the XR Interaction Toolkit samples with XR Hands added as well. I'm also using the default XR Rig that's already setup in the scene. The only thing I changed in the scene is adding the marker tracking script.

@chaseryals219 & @tyler.morris this is really interesting. Lets try seeing if its the unity version. Could y'all try updating to 2022.3.30f1?
Also I will get a zip sent to both of you today with my project so we can see if it works.

@chaseryals219 btw the apk that I sent you guys is the v2.2 samples project compiled with no changes except to set the target SDK version in the Android player settings to 'Automatic'. (Unity v2022.3.30f1) Good to know that the sample apk worked for you at least. Sidney will send you guys a more targeted sample, but it would probably still be useful to see if you can compile and run the example project without issues. You can download the sample project from ML Hub or if you want, I'd be happy to send you a copy of the project.

I am attempting to update my project to 2022.3.30f1 now. Will report my results when I test.

@aheaney To keep testing results consistent, could you send me a copy of the project?

Still getting a pose of 0's on 2022.3.30f1

Magicleap_examples.zip (3.1 MB)
All I did was duplicate a scene in the examples and change the script on it to be the one I posted above. The zip there is without the library folder so you'll need to add the non-unity packages back in manually (magic leap sdk, mrtk things)

Using the provided Zip @aheaney provided, I installed Logcat, added breakpoints at line 103 and 121 in the MarkerTrackingExample script.

Not only did the app continue to produce false detections, hitting breakpoints saying a marker was present when there wasn't one, but I also am still getting pose 0.

Looking at Logcat, I am seeing this repeating a lot

I will now attempt with @sfernandez zip