Camera API Compatibility with OpenXR, OS and Vuforia

[kbabilinski]

I'm not sure which version of the SDK and OpenXR configuration Vuforia supports. But you can use the OpenXR APIs and still access the CVCamera Pose using the legacy callback as long as the perception snapshot feature is enabled and then make sure that the references space of your app matches the one returned by the camera

using this script. Just make sure to enable the Magic Leap Reference Space Feature: Reference Space Overview | MagicLeap Developer Documentation

using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using UnityEngine;
using UnityEngine.XR;
using UnityEngine.XR.Management;
using UnityEngine.XR.OpenXR;
using UnityEngine.XR.OpenXR.Features.MagicLeapSupport;

public class ReferenceSpaceToggle : MonoBehaviour
{
    private bool inputSubsystemValid;
    private XRInputSubsystem inputSubsystem;

    // Start is called before the first frame update
    IEnumerator Start()
    {

       var referenceSpaceFeature = OpenXRSettings.Instance.GetFeature<MagicLeapReferenceSpacesFeature>();
       if (!referenceSpaceFeature.enabled)
       {
           Debug.LogError("Unbounded Tracking Space cannot be set if the OpenXR Magic Leap Reference Spaces Feature is not enabled. Stopping Script.");
           yield break;
       }

       yield return new WaitUntil(() => XRGeneralSettings.Instance != null &&
                                         XRGeneralSettings.Instance.Manager != null &&
                                         XRGeneralSettings.Instance.Manager.activeLoader != null &&
                                         XRGeneralSettings.Instance.Manager.activeLoader.GetLoadedSubsystem<XRInputSubsystem>() != null);
       
       inputSubsystem = XRGeneralSettings.Instance.Manager.activeLoader.GetLoadedSubsystem<XRInputSubsystem>();
       TrackingOriginModeFlags supportedModes = inputSubsystem.GetSupportedTrackingOriginModes();
       
       string supportedSpaces = string.Join("\n",
            ((TrackingOriginModeFlags[])Enum.GetValues(typeof(TrackingOriginModeFlags))).Where((flag) =>
                supportedModes.HasFlag(flag) && flag != TrackingOriginModeFlags.Unknown));
       Debug.Log($"Supported Spaces:{supportedSpaces}");
       
       string currentSpace = inputSubsystem.GetTrackingOriginMode().ToString();
       Debug.Log($"Current Space:{currentSpace}");
       
       inputSubsystemValid = true;

       SetSpace(TrackingOriginModeFlags.Unbounded);
    }

    public void SetSpace(TrackingOriginModeFlags flag)
    {
        if (inputSubsystemValid)
        {
            if (inputSubsystem.TrySetTrackingOriginMode(flag))
            {
                string currentSpace = inputSubsystem.GetTrackingOriginMode().ToString();
                Debug.Log($"Current Space:{currentSpace}");
                inputSubsystem.TryRecenter();
                return;
            }
        }
        Debug.LogError("SetSpace failed to set Tracking Mode Origin to " + flag.ToString());
    }
}

Or like you mentioned, you could use the Pixel Sensor API in your script logic, as long as Vuforia supports OpenXR

[kbabilinski]

I also caught a bug in our documentation of the Pixel Sensor GetPose() function. While we wait for the doc to be updated on the dev portal, here is the updated example

    private void GetSensorPose()
    {
        // Get sensor Pose
        Pose sensorPose = pixelSensorFeature.GetSensorPose(sensorType);

        // Updates the Sensor Pose to be relative to the XR Origin
        if (xrOrigin = FindAnyObjectByType<XROrigin>())
        {
            Vector3 worldPosition = xrOrigin.CameraFloorOffsetObject.transform.TransformPoint(sensorPose.position);
            Quaternion worldRotation = xrOrigin.transform.rotation * sensorPose.rotation;
            // Update the existing pose
            sensorPose = new Pose(worldPosition, worldRotation);
        }

        Debug.Log("Sensor Pose:" + sensorPose);
    }

Mainly, the offset parameter is not used and instead the Sensor Pose is transformed by the XR Origin

[zhhqu0305]

Thanks for your messages! RE permission, I think Spatial Mapping Permission was granted, and that this arised when I tried to configure an ImageOnly camera stream instead of a video stream. Still trying to figure out what exactly is wrong here.

RE OpenXR, I think as soon as I enable OpenXR and attach the Vuforia script to the XR Rig, the app would crash as soon as it's opened. the MRTK3 solution worked, but only on Unity SDK 2.1.0. I can rely on MLCamera + Vuforia without OpenXR for now but would wish to see new samples from Vuforia to be published. I'll try the PixelSensor API then.

Thanks!