Depth camera accuracy on Magic Leap 2

My setup is indoor, wearing ML2 and looking at a hand held tool with infrared reflective spheres. I exported one frame's raw buffer and intrinsics parameters to process in OpenCV following this post to get the point clouds. I noticed that the distance measurements between the spheres are not really accurate relative to the world