Different projection matrices on different devices

Native Development
1

Dear Magic Leap team,
We encountered an issue with the projection matrix, for the virtual cameras, we receive from Magic Leap FrameInfo.
On one device it looks OK (3D models stick to the ground), on another device the model is about 30cm above the ground.

Comparing the projection matrices of both devices shows differences:

Magic Leap A, with good projection matrix

Projection matrix left
2.49784,        0,              0,  0
0,              2.04487,        0,  0
-0.00798269,    -0.00335655,    0,  -1
0,              0,              1,  0

Projection matrix right9
2.48666,        0,              0,  0
0,              2.04036,        0,  0
0.0121354,      -0.0128078,     0,  -1
0,              0,              1,  0

Magic Leap B, with incorrect projection matrix

Projection matrix left
2.49193,        0,              0,  0
0,              2.04364,        0,  0
-0.00595741,    0.00276984,     0,  -1
0,              0,              1,  0

Projection matrix right
2.4886,         0,              0,  0
0,              2.0403,         0,  0
0.0103683,      0.00697562,     0,  -1
0,              0,              1,  0

We are using this code for retrieving the projection matrix:

MLResult result = MLGraphicsBeginFrameEx( mGraphicsClient, &frame_params, &frame_info );
MLMat4f projectionMatrixLeft = frame_info.virtual_cameras[0].projection;
MLMat4f projectionMatrixRight = frame_info.virtual_cameras[1].projection;

Both devices using
ML2 OS version: 1.4.0-dev1
MLSDK version: 1.4.0-dev1
Host OS: macOS

Are we missing something? Why can the projection matrix be that different for two devices?

Thanks in advance,
Richard

2

Hello Richard,

I would expect that the projection matrices might vary slightly due to differences in calibration between devices. Those sets of matrices look reasonable to me at a glance. By the way, note that the position and orientation of the views is contained in the transform matrix on the MLGraphicsVirtualCameraInfo structure.

Does the offset appear in both eyes?
Also, do you notice issues in other applications? (even things like the home menu or web browser) If the camera matrices were wrong, then I might expect that you'd see issues in other applications.

How are you rendering models? Have you written a native application? If so, how you are you determining where to position models in space? The way that you are localizing content in world space would likely explain an offset where content appears between devices.

Best,
Adam

3

Hello Adam,

The c-api meshing sample from the Magic Leap SDK is working fine. As you wrote, it it's not about the device.

We are using native application to render a 3D model. This 3D model was created using a LiDAR scan. For example, a scan of my desk including floor, wall and ceiling.

That 3D model should be perfectly aligned to the environment. However, in our native app on Magic Leap it looks like this 3D model is too small and does not fit environment. Dimensions and scaling of the 3D model are correct. The world pose for that 3D model looks fine. It looks more like an issue with focal length for the virtual camera.

I compared the projection matrix for the meshing sample with ours. It's slightly different, but that might be related to our frame_params we are passing to MLGraphicsBeginFrameEx.

frame_params.surface_scale = 1.0;
frame_params.projection_type = MLGraphicsProjectionType_ReversedInfiniteZ;
frame_params.near_clip = 1.0;
frame_params.focus_distance = 1.0f;

Meshing sample

2.4778s,        0.0000s,        0.0000s,    0.0000s
0.0000s,        2.0428s,        0.0000s,    0.0000s
 -0.0065s,      0.0073s,        -1.0769s,   -1.0000s
0.0000s,        0.0000s,        -0.7692s,   0.0000s

our app

2.48483,        0,              0,          0
0,              2.04859,        0,          0
-0.0108146,     -0.00118778,    0,          -1
0,              0,              1,          0

Thanks,
Richard

4

Just wanted to follow up and let you know that we are investigating this issue. And just to make sure I understand the issue completely, the projection Matrix is taken from the camera that renders the image on the Magic Leap? Additionally, does the offset appear inside the headset, outside of device stream? Do you mind trying to update to the latest version of the OS and factory data resetting your device and seeing if the issue persists.