# XR\_ML\_world\_mesh\_detection timing issue **URL:** \ **Category:** OpenXR\ **Tags:** meshing, openxr\ **Created:** [November 7, 2024, 6:20pm UTC](https://forum.magicleap.cloud/t/xr-ml-world-mesh-detection-timing-issue/4588 "2024-11-07T18:20:30Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![simon.zeni](https://avatars.discourse-cdn.com/v4/letter/s/a183cd/32.png) [@simon.zeni](https://forum.magicleap.cloud/u/simon.zeni)\ **Post date:** [November 7, 2024, 6:20pm UTC](https://forum.magicleap.cloud/t/xr-ml-world-mesh-detection-timing-issue/4588/1 "2024-11-07T18:20:30Z")
Hello there, I'm trying to get the `XR_ML_world_mesh_detection` extension working in my application. I followed the sample code [1](https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#_sample_code_3) provided in the extension with very few modification, and it seems like the mesh detection step is stalling the whole application. As you can see in the log beneath, there seems to be an issue with the timestamp in `xrRequestWorldMeshStateCompleteML` causing the stall. ```auto log:491:11-06 18:50:38.264 13284 13306 V REDACTED: [/REDACTED/mesh_detector.c:118] query mesh info log:492:11-06 18:50:38.264 13284 13306 V REDACTED: [/REDACTED/mesh_detector.c:126] xrRequestWorldMeshStateCompleteML log:493:11-06 18:50:38.264 13284 13306 V REDACTED: [/REDACTED/mesh_detector.c:133] xrRequestWorldMeshStateCompleteML done at 0 log:494:11-06 18:50:38.264 13284 13306 I REDACTED: [/REDACTED/mesh_detector.c:135] completion futureResult: XR_SUCCESS log:496:11-06 18:50:38.264 13284 13306 V REDACTED: [/REDACTED/mesh_detector.c:142] mesh_count 0 meshBlockStateCountOutput 429 log:497:11-06 18:50:38.264 13284 13306 V REDACTED: [/REDACTED/mesh_detector.c:144] realloc mesh_states & mesh_blocks log:498:11-06 18:50:38.264 13284 13306 V REDACTED: [/REDACTED/mesh_detector.c:163] xrRequestWorldMeshStateCompleteML log:556:11-06 18:50:42.173 13284 13306 E com.REDACTED: leapcore/frameworks/perception/data_sources/include/pad/xpad_data_source.h(105) GetClosestTimestampedData(): log:557:11-06 18:50:42.173 13284 13306 E com.REDACTED: ERR: Data Not Found for timestamp: 10416275us, now time: 10421734908us log:558:11-06 18:50:42.174 13284 13306 V REDACTED: [/REDACTED/mesh_detector.c:169] xrRequestWorldMeshStateCompleteML done at 10416275399 ``` The timestamp printed by `GetClosestTimestampedData` seems a little off compared to the `now time`, could this be the issue? The native meshing api doesn't seems to be affected by the same issue, I'm able to get a world mesh without stalling everything. I'm updating the `mesh_detector` between `xrWaitFrame` and `xrBeginFrame`, only if `XrFrameState::shouldRender` is true and the session is visible or focused. I can provide more code if requested. Thanks. **ML2 OS version** : 1.8.0 **MLSDK version** : 1.10 ---
**Author:** ![etucker](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/etucker/32/842_2.png) [@etucker](https://forum.magicleap.cloud/u/etucker)\ **Post date:** [November 7, 2024, 6:24pm UTC](https://forum.magicleap.cloud/t/xr-ml-world-mesh-detection-timing-issue/4588/2 "2024-11-07T18:24:01Z")
---
**Author:** ![etucker](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/etucker/32/842_2.png) [@etucker](https://forum.magicleap.cloud/u/etucker)\ **Post date:** [November 7, 2024, 10:53pm UTC](https://forum.magicleap.cloud/t/xr-ml-world-mesh-detection-timing-issue/4588/3 "2024-11-07T22:53:05Z")
Hi @simon.zeni, Welcome to the Magic Leap 2 Forums. We are so grateful for your engagement here. I am going to try to reproduce this issue using this example and I will report back as soon as I learn more. ---
**Author:** ![aheaney](https://avatars.discourse-cdn.com/v4/letter/a/f08c70/32.png) [@aheaney](https://forum.magicleap.cloud/u/aheaney)\ **Post date:** [November 15, 2024, 8:25pm UTC](https://forum.magicleap.cloud/t/xr-ml-world-mesh-detection-timing-issue/4588/4 "2024-11-15T20:25:13Z")
Hello, Sorry for the delay. We'll see if we can reproduce the performance issue you describe here. If you have a relatively self-contained code sample that you can share, we'd be happy to take a look through your code and try running it. I haven't observed problems after some quick testing with applications that use the OpenXR world mesh apis, but that doesn't mean there isn't an issue. In particular, it might be interesting to take a look at your frame timing loop if you're making calls to the world mesh apis on the same thread. The mesh apis are meant to be async, but there might be something there. By the way, just for sanity, by "The native meshing api doesn't seems to be affected by the same issue" are you referring to the MLMeshing C apis? Were you using the MLMeshing apis in the same OpenXR application, or were you using the ML C apis exclusively? If it's easy enough to do, it might be interesting to compare your implementation using the OpenXR meshing extension to your previous implementation. We can send you a dm or setup a private channel if you'd rather not share code publicly. Best, Adam ---
**Author:** ![simon.zeni](https://avatars.discourse-cdn.com/v4/letter/s/a183cd/32.png) [@simon.zeni](https://forum.magicleap.cloud/u/simon.zeni)\ **Post date:** [November 15, 2024, 9:54pm UTC](https://forum.magicleap.cloud/t/xr-ml-world-mesh-detection-timing-issue/4588/5 "2024-11-15T21:54:27Z")
Hi @aheany, don't worry for the delay. Here are both header and source from my application, beside a few logging headers and the [xr\_linear.h](https://github.com/KhronosGroup/OpenXR-SDK/blob/main/src/common/xr_linear.h) header from the OpenXR-SDK, it should be fairly self contained. [mesh\_detector.zip](https://forum.magicleap.cloud/uploads/short-url/qn15hCZgHEcQ9rQrjyjyIm474L.zip) (2.9 KB) As for frame loop, I update the mesh detector between `xrWaitFrame` and `xrBeginFrame` only if `XrFrameState::shouldRender` is `XR_TRUE` and the session state is either `XR_SESSION_STATE_VISIBLE` or `XR_SESSION_STATE_FOCUSED`. I though about moving the mesh\_detector in its own thread but haven't done it yet. Yes I'm refering to the MLMeshing C api, which I also tried in parallel (exclusively), you can see the issues I found with it [here](https://forum.magicleap.cloud/t/sporadic-stacktrace-with-native-meshing-api/4603). I can also share the code if you'd like. ---
**Author:** ![etucker](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/etucker/32/842_2.png) [@etucker](https://forum.magicleap.cloud/u/etucker)\ **Post date:** [November 19, 2024, 4:40pm UTC](https://forum.magicleap.cloud/t/xr-ml-world-mesh-detection-timing-issue/4588/6 "2024-11-19T16:40:40Z")
Just posting here to let you know that we are still working through your issue. We will report back as soon as we learn more. ---
**Author:** ![etucker](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.magicleap.cloud/etucker/32/842_2.png) [@etucker](https://forum.magicleap.cloud/u/etucker)\ **Post date:** [November 25, 2024, 11:09pm UTC](https://forum.magicleap.cloud/t/xr-ml-world-mesh-detection-timing-issue/4588/7 "2024-11-25T23:09:15Z")
Hi @simon.zeni, We have found where your application has stalled and we are working with our team to find the root cause of this issue. I will report back as soon as we learn more. ---
**Author:** ![simon.zeni](https://avatars.discourse-cdn.com/v4/letter/s/a183cd/32.png) [@simon.zeni](https://forum.magicleap.cloud/u/simon.zeni)\ **Post date:** [November 26, 2024, 1:44am UTC](https://forum.magicleap.cloud/t/xr-ml-world-mesh-detection-timing-issue/4588/8 "2024-11-26T01:44:24Z")
Thanks for keeping me updated @etucker I hope you'll get to the bottom of this soon! ---
**Author:** ![aheaney](https://avatars.discourse-cdn.com/v4/letter/a/f08c70/32.png) [@aheaney](https://forum.magicleap.cloud/u/aheaney)\ **Post date:** [November 27, 2024, 12:12am UTC](https://forum.magicleap.cloud/t/xr-ml-world-mesh-detection-timing-issue/4588/9 "2024-11-27T00:12:30Z")
Hi @simon.zeni , Thanks for your patience with this issue! We tried profiling your code sample and the world mesh api calls were running far longer than would be expected- ![image](https://us1.discourse-cdn.com/flex019/uploads/magicleap1/original/2X/5/53dd94ceb453beb9e5398593d10eb09efa670b8d.png) There is currently an issue where the api calls will hang if you attempt to query for meshes relative to VIEW space. We're reported the issue to our internal teams. In the mean time, would you be able to query for meshes relative to XR\_REFERENCE\_SPACE\_TYPE\_LOCAL or XR\_REFERENCE\_SPACE\_TYPE\_UNBOUNDED\_MSFT and see if you observe similar performance problems? Thanks! Adam ---
**Author:** ![simon.zeni](https://avatars.discourse-cdn.com/v4/letter/s/a183cd/32.png) [@simon.zeni](https://forum.magicleap.cloud/u/simon.zeni)\ **Post date:** [November 27, 2024, 6:59pm UTC](https://forum.magicleap.cloud/t/xr-ml-world-mesh-detection-timing-issue/4588/10 "2024-11-27T18:59:48Z")
Hi @aheaney Using `XR_REFERENCE_SPACE_TYPE_LOCAL` as `meshSpace` seems to work fine. I get a different number of mesh blocks (but not 0). I'll wire up the rendering for the XR mesh detection and see if the meshes make sense. Thanks. ---
**Author:** ![simon.zeni](https://avatars.discourse-cdn.com/v4/letter/s/a183cd/32.png) [@simon.zeni](https://forum.magicleap.cloud/u/simon.zeni)\ **Post date:** [November 29, 2024, 7:51pm UTC](https://forum.magicleap.cloud/t/xr-ml-world-mesh-detection-timing-issue/4588/11 "2024-11-29T19:51:11Z")
Hi again @aheaney At first I though that using `XR_REFERENCE_SPACE_TYPE_LOCAL` would fix the issue on my side. I reworked the `mesh_detector` file to process the vertex data, but I noticed weird a behaviour. First, a lot of block states have the `XR_WORLD_MESH_BLOCK_STATUS_DELETED_ML` for states that I haven't seen before. Then the same blocks have `XR_WORLD_MESH_BLOCK_RESULT_FAILED_ML` as `blockResult`. The rare block states that have a different status also all have `XR_WORLD_MESH_BLOCK_RESULT_FAILED_ML` as `blockResult`. No blocks are rendered by my application. Here are the new and updated `mesh_detector` files [mesh\_detector.zip](https://forum.magicleap.cloud/uploads/short-url/cGBtktEdo8MpVVI6QSYbwQ4f9c6.zip) (3.8 KB) And an execution log of the application [log.txt](https://forum.magicleap.cloud/uploads/short-url/rqjYKBmQ4XTyxP1J7b0kKjtpVdW.txt) (1.9 MB) Thanks. ---
**Author:** ![simon.zeni](https://avatars.discourse-cdn.com/v4/letter/s/a183cd/32.png) [@simon.zeni](https://forum.magicleap.cloud/u/simon.zeni)\ **Post date:** [December 10, 2024, 3:26pm UTC](https://forum.magicleap.cloud/t/xr-ml-world-mesh-detection-timing-issue/4588/12 "2024-12-10T15:26:35Z")
Hi @aheaney, I haven't heard anything from you since my last report. Did you made any progress regarding the deleted/result failed blocks issue? ---
**Author:** ![aheaney](https://avatars.discourse-cdn.com/v4/letter/a/f08c70/32.png) [@aheaney](https://forum.magicleap.cloud/u/aheaney)\ **Post date:** [December 10, 2024, 8:53pm UTC](https://forum.magicleap.cloud/t/xr-ml-world-mesh-detection-timing-issue/4588/13 "2024-12-10T20:53:31Z")
Hi @simon.zeni Sorry for the slow follow up here. At a glance, the results you presented do seem unusual. World mesh OpenXR apis are flexed though our Unreal sdk, so we should be able to determine what's going on. (the sdk is distributed as source btw, you can download the 'Unreal Plugin' package from ML hub if you want to take a look at the world mesh plugin implementation) We'll take a closer look and do some testing with the code sample that you sent over. In the mean time, if you wanted to send over complete source for a runnable application, then that might help accelerate things a bit. Best, Adam ---
**Author:** ![simon.zeni](https://avatars.discourse-cdn.com/v4/letter/s/a183cd/32.png) [@simon.zeni](https://forum.magicleap.cloud/u/simon.zeni)\ **Post date:** [December 11, 2024, 7:31pm UTC](https://forum.magicleap.cloud/t/xr-ml-world-mesh-detection-timing-issue/4588/14 "2024-12-11T19:31:43Z")
Thanks for your quick answer @aheaney, is there a place I can DM or email you? I can send an archive of the source with both the ML and the XR meshing api and build directives. ---
**Author:** ![aheaney](https://avatars.discourse-cdn.com/v4/letter/a/f08c70/32.png) [@aheaney](https://forum.magicleap.cloud/u/aheaney)\ **Post date:** [December 12, 2024, 8:55pm UTC](https://forum.magicleap.cloud/t/xr-ml-world-mesh-detection-timing-issue/4588/15 "2024-12-12T20:55:35Z")
Sure, I'll send you a dm that you can reply to.