I’m developing an ArUco marker tracking system on Magic Leap 2 using C++ with OpenCV. I estimate the marker pose relative to the physical center world-facing camera using solvePnP, and then display a corresponding 3D object in Unity.
In Unity, the main camera represents the HMD origin (i.e., the center between the user’s eyes). To correctly align the virtual marker with the physical one, I need to apply a fixed physical offset between the center world-facing camera and the HMD origin.
At the moment, I’m applying this offset manually in Unity.
My question is:
Is there any official documentation or reliable specification for the physical transform (translation and possibly rotation) between the center world-facing camera and the HMD’s physical origin (as defined by OpenXR’s tracking space)?
Any detail or hardware-level guidance would be very helpful to ensure precise alignment between physical and virtual objects.
You’re right that setting the tracking mode to Unbounded ensures the camera_pose is expressed in the HMD coordinate system (i.e., relative to the user’s head center, as Unity expects).
However, the center-world camera is physically offset from the HMD center.
So even with Unbounded tracking, if you’re estimating marker pose relative to the camera (e.g., using solvePnP), you still need to apply a fixed physical offset from the camera to the HMD center to align virtual objects properly in Unity, no?
This offset reflects the actual position of the camera on the headset — a few centimeters forward, up, or to the side — and should be applied before transforming the marker pose into world space.
To convert the tracked position to world coords we get the camera pose of the frame using MLCVCameraGetFramePose (we still use the old MLCamera Api) before sending it to unity.
