How to override Magic Leap 2 head tracking with external OptiTrack position data?

Unity Development
1

Hi everyone,

I’m trying to use an OptiTrack motion capture system to override the Magic Leap 2’s internal head tracking. The goal is to have the ML2 camera position/rotation controlled entirely by OptiTrack when
tracking data is available, and fall back to ML2’s onboard tracking when OptiTrack is unavailable.

Setup

  • Middleware PC: Unity app receives OptiTrack rigid body data via NatNet (Unicast), then sends position/rotation as JSON over UDP to the ML2
  • Magic Leap 2: Unity app receives UDP packets and attempts to set the camera position
  • Data transmission works: UDP packets arrive correctly, I can see the position values in the logs

The Problem:

No matter what I try, the ML2’s internal head tracking keeps interfering. When I move my head, virtual objects wobble/shake because both tracking systems seem to fight for control. When OptiTrack is not sending data, the view is completely frozen (no tracking at all).

What I’ve Tried

  1. Disabling TrackedPoseDriver component on Main Camera at runtime
    • Result: Still wobbles
  2. Setting camera.transform.position directly in various callbacks:
    • Update() - wobbles
    • LateUpdate() - wobbles
    • Application.onBeforeRender with [BeforeRenderOrder(10000)] - wobbles
    • Camera.onPreCull - only ML2 tracking works, OptiTrack ignored
  3. Moving XR Origin (ML Rig) instead of camera
    • Calculated offset to compensate for local camera position
    • Result: Still wobbles
  4. Changing TrackedPoseDriver Update Type to “Update” only (not “Before Render”)
    • Result: Still wobbles
  5. Stopping XRInputSubsystem completely:
    var inputSubsystem = XRGeneralSettings.Instance.Manager
    .activeLoader.GetLoadedSubsystem();
    inputSubsystem.Stop();
    • Result: Same behavior

Hierarchy

ML Rig (XR Origin)
└── Camera Offset
└── Main Camera (TrackedPoseDriver: Update and Before Render)

Virtual objects are in world space (not parented to camera).

Questions:

How can I completely disable or override the ML2’s head tracking to use external position data from OptiTrack?

Is there a specific API or approach that allows manual control of the camera’s world position without the XR system overriding it?

Any help would be greatly appreciated!

Unity Editor version: 2022.3.67.f2
ML2 OS version: 1.12.0
Unity SDK version: 2.6.0
Host OS: Windows

Error messages from logs (syntax-highlighting is supported via Markdown):

2

You cannot disable or override ML2’s head tracking. The device will show a “head pose lost” notification if the cameras do not have sufficient feature points to track the space.

You can use the Pixel Sensor API to obtain images from the world cameras, however they will continue to be used for tracking.

4

So using OptiTrack’s external tracking to support the internal tracking of the ML2 is not possible? Would be great to use it for large spaces we’re working in.