Eye Camera Intrinsics and Extrinsics on Magic Leap 2

Unity Development
, ,
1

Give us as much detail as possible regarding the issue you’re experiencing:

Unity Editor version:
ML2 OS version: 1.10.0
Unity SDK version: 2.6.0
Host OS: MacOS

Error messages from logs (syntax-highlighting is supported via Markdown):

N/A — this is a general API question

Hi Magic Leap team,

I am working with raw eye-camera data from Magic Leap 2, while another developer on our team handles the Unity application.

Our goal is to use the eye-camera images to estimate physical pupil size and eye rotation. I wanted to ask whether the calibration parameters of the four eye cameras are accessible through the SDK or OpenXR Pixel Sensor API.

Specifically:

Can we obtain the intrinsic parameters for each eye camera, such as focal length, principal point, and distortion coefficients?
Can we obtain the extrinsic parameters of each eye camera relative to the headset or a common coordinate system?
Is there a recommended API or example for accessing these parameters?

I found that the Pixel Sensor API supports the four eye-camera streams, but I could not find clear documentation on whether calibration metadata is available specifically for the eye cameras.

Thank you for any guidance.

2

The OpenXR Pixel Sensor API exposes camera intrinsics for the four eye-camera streams (/pixelsensor/eye/temple/left, /pixelsensor/eye/nasal/left, /pixelsensor/eye/nasal/right, /pixelsensor/eye/temple/right).

Camera poses (extrinsics) are queryable relative to the OpenXR tracking coordinate system. You can query the 6-DoF pose (position and orientation) of each eye camera using either the Pixel Sensor API frame pose query or standard OpenXR perception snapshot feature (MLCVCamera.GetFramePose). Having the camera poses in world/headset coordinates allows you to transform estimated pupil positions or 3D eye rotation vectors directly into the application’s local XR head space.

The recommended approach in Unity using the Magic Leap 2 OpenXR Plugin (MagicLeap.OpenXR.Features.PixelSensors): Enable the Pixel Sensor OpenXR Feature under Project Settings. Declare and request runtime permission for com.magicleap.permission.EYE_CAMERA. Subscribe to one or more of the 4 eye paths (e.g., /pixelsensor/eye/temple/left) using PixelSensorFrameType.Grayscale. When accessing frame metadata from the sensor callback/handle, query PixelSensorPinholeIntrinsics or PixelSensorFisheyeIntrinsics.

C#

using MagicLeap.OpenXR.Features.PixelSensors;

// Example pattern for querying intrinsics from sensor metadata
void OnPixelSensorFrameReceived(PixelSensorFrame frame)
{
    // Query pinhole or fisheye intrinsics metadata
    if (frame.TryGetMetadata(PixelSensorMetadataType.PixelSensorPinholeIntrinsics, out PixelSensorPinholeIntrinsics intrinsics))
    {
        Vector2 focalLength = intrinsics.FocalLength;
        Vector2 principalPoint = intrinsics.PrincipalPoint;
        float[] distortion = intrinsics.Distortion;
    }
    
    // Extrinsics / Camera Pose
    Pose cameraPose = frame.Pose; // Pose relative to OpenXR reference space
}