# Depth camera accuracy on Magic Leap 2 **URL:** \ **Category:** Unity Development\ **Tags:** development, unity, depth-camera\ **Created:** [June 24, 2024, 8:12pm UTC](https://forum.magicleap.cloud/t/depth-camera-accuracy-on-magic-leap-2/3695 "2024-06-24T20:12:22Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![htn002](https://avatars.discourse-cdn.com/v4/letter/h/7993a0/32.png) [@htn002](https://forum.magicleap.cloud/u/htn002)\ **Post date:** [June 24, 2024, 8:12pm UTC](https://forum.magicleap.cloud/t/depth-camera-accuracy-on-magic-leap-2/3695/1 "2024-06-24T20:12:22Z")
Hello, I'm using GetLatestDepthData from MLDepthCamera API to get the depth information. However, I noticed that the depth data is not really accurate based on what I've measured (I have 2 retroreflective objects with different depth but it gave me same depth value). Is it the known limitations of the depth camera or is there anything I can configure on the API? Thank you! ---
**Author:** ![sfernandez](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/sfernandez/32/747_2.png) [@sfernandez](https://forum.magicleap.cloud/u/sfernandez)\ **Post date:** [June 24, 2024, 9:25pm UTC](https://forum.magicleap.cloud/t/depth-camera-accuracy-on-magic-leap-2/3695/2 "2024-06-24T21:25:42Z")
Hi @htn002 welcome to the forums. I would like to know more about your setup physically, this will help me trouble shoot the behavior you are seeing as well as help me give context to any other engineers I happen to contact about this. For the device, which os build are you using? Also is the device being held stationary in a rig, or worn, for this test? For the environment, where is this test being done, indoors, outdoors, and what are the lighting conditions in that space? For the targets, what are they? how big are they, and how far away are they from the device and from each other? Lastly software environment, could you please let me know what version of unity, along with which sdk you are using. If there is a script that you are using for your testing and able to share that would be helpful too. -Sidney ---
**Author:** ![htn002](https://avatars.discourse-cdn.com/v4/letter/h/7993a0/32.png) [@htn002](https://forum.magicleap.cloud/u/htn002)\ **Post date:** [June 25, 2024, 2:54pm UTC](https://forum.magicleap.cloud/t/depth-camera-accuracy-on-magic-leap-2/3695/3 "2024-06-25T14:54:33Z")
My setup is indoor, wearing ML2 and looking at a hand held tool with infrared reflective spheres. I exported one frame's raw buffer and intrinsics parameters to process in OpenCV following [this post](https://forum.magicleap.cloud/t/processing-the-depth-frames/3284/13) to get the point clouds. I noticed that the distance measurements between the spheres are not really accurate relative to the world ---
**Author:** ![sfernandez](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/sfernandez/32/747_2.png) [@sfernandez](https://forum.magicleap.cloud/u/sfernandez)\ **Post date:** [June 25, 2024, 3:27pm UTC](https://forum.magicleap.cloud/t/depth-camera-accuracy-on-magic-leap-2/3695/4 "2024-06-25T15:27:46Z")
Ok I think the problem may be the spheres. If they are the retroreflective "passive marker spheres" like you would use for multi-camera 3d tracking you will likely get unreliable results. The ML2 uses a Time of Flight Depth Sensor, and ToF sensors can be confused by highly reflective surfaces and especially retroreflective surfaces. Is there a hard requirement to measure using those markers or were they used out of convenience? ---
**Author:** ![htn002](https://avatars.discourse-cdn.com/v4/letter/h/7993a0/32.png) [@htn002](https://forum.magicleap.cloud/u/htn002)\ **Post date:** [June 25, 2024, 4:48pm UTC](https://forum.magicleap.cloud/t/depth-camera-accuracy-on-magic-leap-2/3695/5 "2024-06-25T16:48:09Z")
Yes, using retroreflective spheres is a must, but the surface material of the spheres can be flexible ---
**Author:** ![sfernandez](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/sfernandez/32/747_2.png) [@sfernandez](https://forum.magicleap.cloud/u/sfernandez)\ **Post date:** [June 25, 2024, 4:52pm UTC](https://forum.magicleap.cloud/t/depth-camera-accuracy-on-magic-leap-2/3695/6 "2024-06-25T16:52:24Z")
Perhaps if you can explain more of your use case then I can think of another way, since the surface being retroreflective is what is likely causing the depth confusion. ---
**Author:** ![htn002](https://avatars.discourse-cdn.com/v4/letter/h/7993a0/32.png) [@htn002](https://forum.magicleap.cloud/u/htn002)\ **Post date:** [June 25, 2024, 6:17pm UTC](https://forum.magicleap.cloud/t/depth-camera-accuracy-on-magic-leap-2/3695/7 "2024-06-25T18:17:52Z")
The use case I'm working on is to measure the distance between two markers using the depth camera. I believe there may be some confusion arising from the depth camera. Would you mind reviewing the converter code to make sure there's nothing arising from the undistortion part? ```auto def camera_space_converter(img_x, img_y, depth): # Camera focal_length = np.array([365.76, 365.76]) principal_point = np.array([265.92, 240.93]) resolution = np.array([544, 480]) # Distortion coefficients K1, K2, K3 = -0.0865550637245178, -0.0201911050826311, -0.00704800290986896 P1, P2 = 0.000115656432171818, -0.000655817857477814 # Camera to world transformation cam_to_world_mat = np.array([ [0.25, 0.11, -0.96, -0.28], [0.01, 0.99, 0.11, -0.40], [0.97, -0.04, 0.24, 0.17], [0.00, 0.00, 0.00, 1.00] ]) uv = np.array([img_x, img_y]) / np.array(resolution) offset_from_center = uv - np.array([0.5, 0.5]) r2 = np.dot(offset_from_center, offset_from_center) r4 = r2 * r2 r6 = r2 * r4 radial_distortion_correction = offset_from_center * (1 + K1 * r2 + K2 * r4 + K3 * r6) tangential_distortion_correction_x = 2 * P1 * offset_from_center[0] * offset_from_center[1] + P2 * (r2 + 2 * offset_from_center[0] * offset_from_center[0]) tangential_distortion_correction_y = 2 * P2 * offset_from_center[0] * offset_from_center[1] + P1 * (r2 + 2 * offset_from_center[1] * offset_from_center[1]) tangential_distortion_correction = np.array([tangential_distortion_correction_x, tangential_distortion_correction_y]) corrected_uv = radial_distortion_correction + tangential_distortion_correction + np.array([0.5, 0.5]) camera_point = (corrected_uv * resolution - principal_point) / focal_length camera_point = np.append(camera_point, 1) camera_point = camera_point/np.linalg.norm(camera_point) * depth world_point = cam_to_world_mat @ np.append(camera_point, 1) return world_point[:3] ``` ---
**Author:** ![tkCADS](https://avatars.discourse-cdn.com/v4/letter/t/74df32/32.png) [@tkCADS](https://forum.magicleap.cloud/u/tkCADS)\ **Post date:** [February 11, 2025, 2:34pm UTC](https://forum.magicleap.cloud/t/depth-camera-accuracy-on-magic-leap-2/3695/8 "2025-02-11T14:34:58Z")
Hi @htn002 , did you ever get the tool tracking to work using this approach with the infrared reflective markers and OpenCV?