Vuforia cannot be used together with ML2 Marker Tracking

zhhqu0305 · August 12, 2023, 8:14pm

When I use Vuforia together with ML2 Marker Tracking, the vuforia engine got stopped after marker tracker starts. I tried to use delayed initialization and initialize vuforia after marker tracker starts but the problem remains. Code I'm using:

using System.Collections.Generic;
using System.Text;
using UnityEngine;
using UnityEngine.XR.MagicLeap;
using Vuforia;

public class MarkerTrackerExample : MonoBehaviour
{
    public float QrCodeMarkerSize = 0.22f;
    public float ArucoMarkerSize = 0.1f;
    public VuforiaBehaviour vuforiaBehaviour;
    public MLMarkerTracker.MarkerType Type = MLMarkerTracker.MarkerType.QR;
    public MLMarkerTracker.ArucoDictionaryName ArucoDict = MLMarkerTracker.ArucoDictionaryName.DICT_5X5_100;
    public MLMarkerTracker.Profile Profile = MLMarkerTracker.Profile.Default;

    private Dictionary<string, GameObject> _markers = new Dictionary<string, GameObject>();
    private ASCIIEncoding _asciiEncoder = new System.Text.ASCIIEncoding();


#if UNITY_ANDROID
    private void OnEnable()
    {
        MLMarkerTracker.OnMLMarkerTrackerResultsFound += OnTrackerResultsFound;
    }

    private void Start()
    {
        MLMarkerTracker.TrackerSettings trackerSettings = MLMarkerTracker.TrackerSettings.Create(
            true, Type, QrCodeMarkerSize, ArucoDict, ArucoMarkerSize, Profile);
        _ = MLMarkerTracker.SetSettingsAsync(trackerSettings);
        Debug.Log("Marker Tracker Started!");
        VuforiaApplication.Instance.Initialize();
        Debug.Log("Vuforia Manually Initialized!");
        vuforiaBehaviour.enabled = true;
        Debug.Log("Vuforia Behaviour Enabled!");

    }

    private void OnDisable()
    {
        MLMarkerTracker.OnMLMarkerTrackerResultsFound -= OnTrackerResultsFound;
    }

    private void OnTrackerResultsFound(MLMarkerTracker.MarkerData data)
    {
        string id = "";
        float markerSize = .01f;

        switch (data.Type)
        {
            case MLMarkerTracker.MarkerType.QR:
                id = _asciiEncoder.GetString(data.BinaryData.Data, 0, data.BinaryData.Data.Length);
                markerSize = QrCodeMarkerSize;
                break;
            default:
                break;
        }

        if (!string.IsNullOrEmpty(id))
        {
            Debug.Log("Marker ID: " + id);
            Debug.Log("S6: Marker Pose: " + data.Pose.position);
            Debug.Log("S6: Marker Rotation: " + data.Pose.rotation);
            if (_markers.ContainsKey(id))
            {
                GameObject marker = _markers[id];
                marker.transform.position = data.Pose.position;
                marker.transform.rotation = data.Pose.rotation;
            }
            else
            {
                //Create a primitive cube
                GameObject marker = GameObject.CreatePrimitive(PrimitiveType.Cube);
                //Render the cube with the default URP shader
                marker.AddComponent<Renderer>();
                marker.GetComponent<Renderer>().material = new Material(Shader.Find("Universal Render Pipeline/Lit"));
                marker.transform.position = data.Pose.position;
                marker.transform.rotation = data.Pose.rotation;
                marker.transform.localScale = new Vector3(markerSize, markerSize, markerSize);
                _markers.Add(id, marker);
            }
        }

    }
#endif
}

The log I get:

08-12 15:12:00.352741  3349  7813 W amdCamera3VirtualCamera: ============= Dump Streams =============
08-12 15:12:00.352763  3349  7813 W amdCamera3VirtualCamera: Width       : 2048
08-12 15:12:00.352767  3349  7813 W amdCamera3VirtualCamera: Height      : 1536
08-12 15:12:00.352770  3349  7813 W amdCamera3VirtualCamera: Format      : 35
08-12 15:12:00.352773  3349  7813 W amdCamera3VirtualCamera: State       : 1
08-12 15:12:00.352776  3349  7813 W amdCamera3VirtualCamera: stream Id   : 0
08-12 15:12:00.352779  3349  7813 W amdCamera3VirtualCamera: stream Type : 2
08-12 15:12:00.352781  3349  7813 W amdCamera3VirtualCamera: ==================================
08-12 15:12:00.353389  4992  5174 I Marker Tracking Service: nova/frameworks/services/perception_services/markertracking_svc/service/src/marker_tracking_manager.cpp(265) InitializeAndStartCamera():
08-12 15:12:00.353389  4992  5174 I Marker Tracking Service: INF: MarkerTrackingManager::InitializeAndStartCamera: Started Cam, success=true
08-12 15:12:00.353420  4992  5174 I Marker Tracking Service: nova/frameworks/services/perception_services/markertracking_svc/service/src/zbar_detector.cpp(70) ZBarDetector():
08-12 15:12:00.353420  4992  5174 I Marker Tracking Service: INF: Setting config for: 64
08-12 15:12:00.353430  4992  5174 I Marker Tracking Service: nova/frameworks/services/perception_services/markertracking_svc/service/src/marker_tracking_manager.cpp(478) CreateDetectors():
08-12 15:12:00.353430  4992  5174 I Marker Tracking Service: INF: MarkerTrackingManager::ParseSettings: # detectors created 1, vcam: 1, wcams: 0
08-12 15:12:00.354161  3414  4757 I cvip-ch : Aug 12 19:11:53 0000007019254000342  Q6.02 00097 FusionQ6Job     [q6/src/wr_q6_blocks_fusion.cpp:00016]:I:: :wr:fusion: Running depth fusion on Q6
08-12 15:12:00.358561 26635 26905 E Unity   : Vuforia Engine has stopped due to an error: The operating system has dropped the camera device used by Vuforia Engine
08-12 15:12:00.358561 26635 26905 E Unity   : Vuforia.Internal.Core.Engine:onEngineErrorHandler(VuEngineError, IntPtr)
08-12 15:12:00.358561 26635 26905 E Unity   : 
08-12 15:12:00.360243  3349  3460 V amdCamera3MetaArb: Cam[1] : assigning the request metadata
08-12 15:12:00.361542 26635 26656 I AR      : Deactivating observer...
08-12 15:12:00.361576 26635 26656 I AR      : Deactivated observer with ID '1'
08-12 15:12:00.361581 26635 26656 I AR      : Deactivating observer...SUCCESS
08-12 15:12:00.361756  3349  3460 V amdCamera3MetaArb: Cam[1] : assigning the request metadata
08-12 15:12:00.362706 26635 26656 I AR      : Destroying observer...
08-12 15:12:00.362736 26635 26656 I AR      : Destroyed observer with ID '1'
08-12 15:12:00.362740 26635 26656 I AR      : Destroying observer...SUCCESS
08-12 15:12:00.362764 26635 26656 I AR      : Destroying observer...
08-12 15:12:00.362784 26635 26656 I AR      : Destroyed observer with ID '2'
08-12 15:12:00.362789 26635 26656 I AR      : Destroying observer...SUCCESS
08-12 15:12:00.362798 26635 26656 I AR      : Destroying observer...
08-12 15:12:00.362808 26635 26656 I AR      : Destroyed observer with ID '3'
08-12 15:12:00.362813 26635 26656 I AR      : Destroying observer...SUCCESS
08-12 15:12:00.363117  3349  3460 V amdCamera3MetaArb: Cam[1] : assigning the request metadata
08-12 15:12:00.365093 26635 26656 I AR      : Destroying Vuforia Engine instance...
08-12 15:12:00.368689  3349  3460 V amdCamera3MetaArb: Cam[1] : assigning the request metadata
08-12 15:12:00.369944  3349  3460 V amdCamera3MetaArb: Cam[1] : assigning the request metadata
08-12 15:12:00.370561  3414  4757 I cvip-ch : Aug 12 19:11:53 0000007019285458744 ARM.01 00796 sc_isp_ctrl     [vice/arm/src/sensor_core_isp.c:00939]:I:: sc_stream_isp_ctrl_thread: Control Req - id:0xffffff08 req:0x80c00fc0
08-12 15:12:00.370848 26635 26656 I AR      : Destroying Vuforia Engine instance...SUCCESS
08-12 15:12:00.371291  3349  3460 V amdCamera3MetaArb: Cam[1] : assigning the request metadata
08-12 15:12:00.372474 26635 26656 I Unity   : Vuforia Deinitialized
08-12 15:12:00.372474 26635 26656 I Unity   : Vuforia.Internal.Core.Engine:Deinit()
08-12 15:12:00.372474 26635 26656 I Unity   : Vuforia.Internal.Core.Engine:UpdateState()
08-12 15:12:00.372474 26635 26656 I Unity   : System.Reflection.RuntimeMethodInfo:Invoke(Object, BindingFlags, Binder, Object[], CultureInfo)
08-12 15:12:00.372474 26635 26656 I Unity   : System.Reflection.MethodBase:Invoke(Object, Object[])
08-12 15:12:00.372474 26635 26656 I Unity   : System.Delegate:DynamicInvokeImpl(Object[])
08-12 15:12:00.372474 26635 26656 I Unity   : Vuforia.Utility.ExtensionMethods.DelegateHelper:InvokeDelegate(Delegate, Object[])
08-12 15:12:00.372474 26635 26656 I Unity   :

I'm also attching the logfile here:
logcat.zip (144.7 KB)

kbabilinski · August 14, 2023, 7:24pm

This issue might be due to the fact that Vuforia and the Marker Tracking feature use the CV camera by default. While you are not able to change the camera that is used by Vuforia, you are able to set the Marker Tracker to use the world camera's instead of the CV camera. To do this use the Large_FOV profile instead of the default one.

Topic		Replies	Views
Use of Vuforia Engine along with Magic Leap Marker Tracking Vuforia Engine marker-tracker	6	104	July 31, 2024
Vuforia and AR Foundation don't work together Vuforia Engine development , unity , vuforia	1	309	March 31, 2023
Vuforia Engine Not Working with New ML2 SDK Unity Development vuforia	7	46	January 14, 2025
Vuforia not working Vuforia Engine development , unity	13	811	October 31, 2022
Vuforia Image Target works in Unity Play Mode (Webcam) but no track on ML2 (and no errors) Vuforia Engine	7	1008	October 6, 2023

Vuforia cannot be used together with ML2 Marker Tracking

Related topics