Tracking Quality Regression Noticed in Unity App on OS 1.10.0

Unity Development
, ,
1

Hello,

I'm working on developing an app in Unity, and we have noticed that the virtual environment will jitter when walking around the space. By jitter, I mean that the virtual content will briefly become unanchored, move with the headset's motion, and then correct itself and snap back into place. One individual jitter only lasts for a fraction of a second, and they seem to vary in frequency. They are common enough to where I will see them nearly every time I chose to walk in my space.

We have noticed this issue on OS version 1.10.0. We are using the Unity MLSDK version 2.5.0. When we roll back to OS versions 1.8 and 1.9 the jittering does not occur. Could this perhaps be related to the Spaces system? We also notice that the Spaces app is not as performant on 1.10. I have also checked the device's logs via the Magic Leap Hub for any indications of what could be causing this, but I couldn't find anything.

Is this a known issue, and is there any way of preventing this on the latest OS? Or is there a plan to address this in an upcoming release?

Thanks!

2

Do you mind sharing device logs or a bug report from the device after you notice the issue? In your application do you mind adding logs to identify when the app searches for Anchors or queries localization status?

3

Sure, here's the device logs from a session where there was a lot of consistent jittering. I have always had some logs to identify anchor queries and current localization queries, but I made sure to update them.

JitterLogs.txt (1.5 MB)

We only query anchors on a specific condition after the callback of
MagicLeapLocalizationMapFeature.OnLocalizationChangedEvent. That condition is if the headset has the status "Localized" and we aren't already localized to the current space. We only ever call MagicLeapLocalizationMapFeature.GetLatestLocalizationMapData once on startup. Otherwise we are saving the current localization data when it is received in the callback of the localization changed event.

When capturing these logs, I kept an eye out for any statements related to localization of our app, but didn't see any. The jitter was near constant at that time. I also ensured our framerate was at a consistent 60 fps at the time.

Log statements that I added:

  • "Querying spatial anchors" (this is the one called when localization changes and criteria are met to query anchors)
  • "New LocalizationData received: {newData mapUUID} name: {newData mapName}" (this happens every time localization is changed)
  • "Running startup localization query", "Startup query success", "Startup query failure" (These run on startup in a coroutine. If we can not get latest localization data, it fails and waits 2 seconds until trying again. Stops loop when successful)
  • "Updated world root from tracking origin change" (This happens if the world root is moved due to a change in the tracking origin via XRInputSubsystem.trackingOriginUpdated
4

Thank you for that information. Did I understand correctly that you noticed that performance was worse in the application regardless of the Localization API calls?

Are you running any other services in your application such as Meshing? Would it be possible for you to test your application and try to isolate the systems that are running to pinpoint what might be causing the slowdown. For example running the App without Marker Tracking.

Do you know if the Segmented Dimmer Render Feature is present? Have to considered removing the feature from your Render Pipeline if it is present?

Lastly, would it be possible to update your application to use OpenXR instead of MLSDK?

5

Yes, the jitter gets worse independently of the Localization API calls. We are also using a few different XR features including Meshing, Marker Tracking, Eye Tracking, and Hand Tracking. We are not using Segmented Dimmer. Also, we are using OpenXR calls over the legacy MLSDK calls for the various XR features.

One thing I do notice is that when marker tracking is running (searching for an AprilTag with a single detector) the jittering gets a lot worse, and subsides when marker tracking stops. I'm still looking for other cases that are causing the jitter to increase, but could this be indicative of a particular issue?

6

Unfortunately, I haven't been able to reproduce it on my end so far so I can't be sure. Have you tried using any of the profiling tools to like Perfetto, Unity, RGP?

You may want to try building up an example from the Unity Example project and slowly combine the different features into one scene and see when the issue becomes noticeable.