# Detecting Spherical Passive Reflective IR Markers **URL:** \ **Category:** Unity Development\ **Tags:** feature-request, depth-camera\ **Created:** [January 27, 2025, 4:55pm UTC](https://forum.magicleap.cloud/t/detecting-spherical-passive-reflective-ir-markers/5095 "2025-01-27T16:55:45Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![tkCADS](https://avatars.discourse-cdn.com/v4/letter/t/74df32/32.png) [@tkCADS](https://forum.magicleap.cloud/u/tkCADS)\ **Post date:** [January 27, 2025, 4:55pm UTC](https://forum.magicleap.cloud/t/detecting-spherical-passive-reflective-ir-markers/5095/1 "2025-01-27T16:55:45Z")
**Unity Editor version** : 6000.0.23f1 **ML2 OS version** : 1.11.0 **Unity SDK version** : 2.6.0-pre.R15 **Host OS** : (Windows/MacOS) Windows 11 Hi everyone, I’m looking to track spherical passive infrared reflective markers, similar to the implementation described in this project for the HoloLens 2: [DINO-Unity on GitHub](https://github.com/HL2-DINO/DINO-Unity?tab=readme-ov-file). Specifically, I’m wondering if it’s possible to use the Depth Sensor to detect the IR reflective material on these markers. Can the Depth Sensor be leveraged for this purpose? If not, are there alternative approaches you’d recommend for tracking such markers? I tried the Unity Example projects for WorldCamera and DepthCamera with different configurations without any luck. Any suggestions, guidance, or resources would be greatly appreciated. Thank you in advance! ---
**Author:** ![kbabilinski](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/kbabilinski/32/19_2.png) [@kbabilinski](https://forum.magicleap.cloud/u/kbabilinski)\ **Post date:** [January 27, 2025, 5:03pm UTC](https://forum.magicleap.cloud/t/detecting-spherical-passive-reflective-ir-markers/5095/2 "2025-01-27T17:03:08Z")
Hi @tkCADS I’m not directly familiar with the project that you shared but you should be able to obtain the raw image from the ToF sensor and use it to detect the reflective IR markers. ## Note, we have had feedback that the PixelSensor API does not provide the flexibility that some developers need since you are **NOT** able to obtain the Raw Image and the Filtered image data at the same time. Here is some additional information about the Depth Sensor Configurations: | Capability | Description | | --- | --- | | Range Modes | **Long Range** (1m - 5m): Operates at 1 or 5 fps | | | **Short Range** (0.20m - 1.00m): Operates at 5 or 30 fps (60 fps is supported but can cause overheating when running for extended periods) | | Resolution | 544 x 480 pixels | | Data Options | Developers can query **Depth Data** or **Raw Depth Data** | | Exposure Adjustment | Developers can adjust the **Exposure Time** | | Depth Range | Developers can clamp the depth range for either mode | Depending on the selected output the following Metadata is available for each frame: | Configuration | Metadata | | --- | --- | | Depth 32 Output | - Depth Confidence Buffer - Depth Flag Buffer - Exposure Time - Pinhole Camera Model | | Raw Depth Output | - Depth Frame Illumination - Exposure Time - Pinhole Camera Model | _The Sensor’s Position and Rotation can be queried regardless of configuration_ ---
**Author:** ![tkCADS](https://avatars.discourse-cdn.com/v4/letter/t/74df32/32.png) [@tkCADS](https://forum.magicleap.cloud/u/tkCADS)\ **Post date:** [January 28, 2025, 6:49am UTC](https://forum.magicleap.cloud/t/detecting-spherical-passive-reflective-ir-markers/5095/3 "2025-01-28T06:49:59Z")
Thanks for the quick and detailed response! Just to clarify, I am looking for a way to achieve an infrared response that allows me to clearly distinguish between the infrared reflective materials and non-reflective materials. To give you a better idea of what I’m aiming for, I’ve attached an image that illustrates my desired outcome: ![image](https://us1.discourse-cdn.com/flex019/uploads/magicleap1/original/2X/1/16345e3f64ad35ba73f275a559d2660aaaf96372.png) In your opinion, which configuration and metadata response would be best suited for this purpose? Thanks again! ---
**Author:** ![kbabilinski](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/kbabilinski/32/19_2.png) [@kbabilinski](https://forum.magicleap.cloud/u/kbabilinski)\ **Post date:** [January 28, 2025, 4:52pm UTC](https://forum.magicleap.cloud/t/detecting-spherical-passive-reflective-ir-markers/5095/4 "2025-01-28T16:52:30Z")
I think that would be _Depth Frame Illumination_, but I'm not familiar with this subject. If you have a Magic Leap 2 device you can see the latest Unity Example Project , it comes with a Pixel Sensor Example scene. Alternatively we have a Pixel Sensor Section on the Developer Documentation. ---
**Author:** ![tkCADS](https://avatars.discourse-cdn.com/v4/letter/t/74df32/32.png) [@tkCADS](https://forum.magicleap.cloud/u/tkCADS)\ **Post date:** [February 7, 2025, 8:44am UTC](https://forum.magicleap.cloud/t/detecting-spherical-passive-reflective-ir-markers/5095/5 "2025-02-07T08:44:34Z")
I was able to work out the IR response using **DepthRaw** —thanks for the guidance! However, as you previously mentioned, it’s not possible to access both **DepthRaw** and **Depth32** simultaneously. I have a few questions about potential workarounds: - Is there a way to extract depth data from **DepthRaw**? - Can the **Depth32** buffer provide an IR response, perhaps by leveraging the **DepthFrameIllumination** option somehow? - I assume switching between the two buffers every frame would not work due to latency—can you confirm? - Are there any alternative methods to obtain both Depth and IR response data at the same time? ---
**Author:** ![kbabilinski](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/kbabilinski/32/19_2.png) [@kbabilinski](https://forum.magicleap.cloud/u/kbabilinski)\ **Post date:** [February 17, 2025, 4:45pm UTC](https://forum.magicleap.cloud/t/detecting-spherical-passive-reflective-ir-markers/5095/6 "2025-02-17T16:45:38Z")
I think it would be impossible for you to obtain depth using the DepthRaw image because: Raw Depth Image : This is the raw depth camera sensor data captured with the depth camera illumination and corresponds to the amount of total light incident on the sensor. vs Depth Image: stores the Depth Data from the Depth Camera. Depth is represented in meters While not on our roadmap, I will put in a feature request to the Voice of Customer team based on your requirements. ---
**Author:** ![alessandro.albanesi](https://avatars.discourse-cdn.com/v4/letter/a/c77e96/32.png) [@alessandro.albanesi](https://forum.magicleap.cloud/u/alessandro.albanesi)\ **Post date:** [April 14, 2025, 3:59pm UTC](https://forum.magicleap.cloud/t/detecting-spherical-passive-reflective-ir-markers/5095/7 "2025-04-14T15:59:24Z")
Good afternoon everyone, I’m following up on this topic to ask whether it’s possible to stream both the **raw IR data** and the **processed depth data in meters** from the sensor—similar to what was possible on the HoloLens 2. If simultaneous streaming of both data types isn’t supported, is there a way to derive the depth in meters from the **raw sensor data** alone? Any insights or pointers would be greatly appreciated! ---
**Author:** ![tom](https://avatars.discourse-cdn.com/v4/letter/t/3d9bf3/32.png) [@tom](https://forum.magicleap.cloud/u/tom)\ **Post date:** [May 1, 2025, 5:26pm UTC](https://forum.magicleap.cloud/t/detecting-spherical-passive-reflective-ir-markers/5095/8 "2025-05-01T17:26:24Z")
Is there an update yet, did you achieve tracking the IR markers? ---
**Author:** ![magicbot](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/magicbot/32/2470_2.png) [@magicbot](https://forum.magicleap.cloud/u/magicbot)\ **Post date:** [May 1, 2025, 5:28pm UTC](https://forum.magicleap.cloud/t/detecting-spherical-passive-reflective-ir-markers/5095/9 "2025-05-01T17:28:24Z")
Hi All, The ability to query both depth and raw image from the Depth Sensor is a feature that is not currently on our roadmap. Unfortunately there is no way to obtain depth data from the raw image. ---
**Author:** ![hmmwhatt](https://avatars.discourse-cdn.com/v4/letter/h/8baadc/32.png) [@hmmwhatt](https://forum.magicleap.cloud/u/hmmwhatt)\ **Post date:** [June 20, 2025, 7:53pm UTC](https://forum.magicleap.cloud/t/detecting-spherical-passive-reflective-ir-markers/5095/10 "2025-06-20T19:53:42Z")
hello Ive been working on the same project and im new to unity dev so were you able to achieve this and if so can you share me the git repo ---
**Author:** ![GspLi](https://avatars.discourse-cdn.com/v4/letter/g/35a633/32.png) [@GspLi](https://forum.magicleap.cloud/u/GspLi)\ **Post date:** [July 22, 2025, 6:17am UTC](https://forum.magicleap.cloud/t/detecting-spherical-passive-reflective-ir-markers/5095/11 "2025-07-22T06:17:19Z")
> The ability to query both depth and raw image from the Depth Sensor is a feature that is not currently on our roadmap. Unfortunately there is no way to obtain depth data from the raw image. Majority of precision tracking depends on getting the RawDepth and Depth data simultaneously. From a development perspective, that’s not optimistic for us. I mean, Hololens 2 did it 6 years ago. Would it be possible to raise this matter again?