Feedback on OpenXR marker tracking

General
,
1

Hi folks,

I have been messing around with the new OpenXR marker tracking and even went so far as to develop a Reality Collective Service Framework service for it, abstracting the actual platform specific tracking code into something that allows a QR code tracking app to run on either HoloLens 2 and Magic Leap unchanged. I published a demo project on GithHub that shows it displaying game objects positioned on QR codes. This allowed me to compare HoloLens 2 and Magic Leap 2 QR code tracking, and I made the following observations:

  1. HoloLens is faster in initial detecting a QR code and determining its position.
  2. HoloLens is also a bit faster in following moving QR codes
  3. However, if you do a video recording while tracking, Magic Leap is faster. But:
  4. If you try to do a video recording during marker tracking, Magic Leap shows a uniform black background, while HoloLens shows the camera view.
  5. HoloLens seems be a bit better and show a more stable view. When you look a code at 90° angle the performance is pretty much the same, but when you are looking at a smaller angle, Magic Leap has more difficulties finding the right display angle and tends to rotate the marker towards the camera in stead of keeping it at the marker angle.

Also, the only detector profile I have been able to make show up markers is the Default profile.

In its current form is already quite usable, but I wonder if we can expect improvements in performance and stability in the coming versions. I'll be happy to test and report back :slight_smile:

  • Unity 2022.3.40f1
  • OS 1.8.0
  • SDK 2.3.0
  • ML MRTK 1.1.0
  • Demo project
2 Likes
2

Hi @localjoost,

Thank you for providing thorough feedback. I am surprised to hear that the markers do not work outside of the default profile. I'll be sure to provide this feedback to our team. May I ask, when and how are you changing the marker tracking profile?

3

@localjoost can you try using the Large FoV profile using this APK?
com.magicleap.capi.xr.sample.marker_tracking.apk.zip (3.6 MB)

4

Thanks @localjoost for the valuable feedback. We'll be looking into especially the part with the profile.

Did you exclusively test with QR codes or also ArUco markers and April Tags?

5

@Hi @etucker

I wrote a little project to test the cross-platform QR code tracking. Configuration happens like this, and then I recompiled the project

6

Hi @ababilinski

I will, as soon as I have access to my Magic Leap 2 again. Currently off-site

7

Hi @BenjiRenji ,

No, I exclusively tested with QR coded to be able to compare apples to apples, as HoloLens cannot natively track ArUco markers. Do you think/know those work better than QR codes on ML2?

8

Hey @localjoost ,

Regarding the black background in captures, I was noticing in another project that utilizes URP that if HDR is enabled in the main URP asset, I was getting black background as well. Disabling HDR seems to fix that, so just wondering if your project is also URP and if you have HDR enabled?

This project I was testing with was utilizing Spectator, which uses ArUco markers (not QR, but still) and I think I was able to attempt capture during the marker tracking step and it seemed fine (got passthrough background instead of black background).

image

1 Like
9

Some aspects like pose accuracy should work better with ArUco markers or April Tags, yes.

10

Confirmed. This works. Movie becomes dark (no background) during Unity splash screen but background becomes visible as soon as the app starts. Thanks!

11

I have upgraded to 1.9.0 and the QR tracking now seems to work regardless of what profile I choose. I still have the feeling the default works the best

12

So I tested a bit with Aruco markers. Those seem to work a bit faster indeed, but that is hardly surprising as their payload is a lot lower (I used the 5x5) - the can only contain numbers, whereas QR codes can contain a lot more

Funny thing: tried codes 12, 42 and 99. 12 and 42 were recognized instantly, 99 never.

13

There are four different variations of the 5X5 dictionary: 50, 100, 250 and 1000.
They indicate how many IDs will be detected (the smaller the faster (slightly)), so if 99 wasn't detected it's most likely you used the 5x5_50 dictionary in your test.

Ultimately, what leads to latency is the operations done on the image to detect the marker candidates. Since ArUco marker detection works completely different than the QR code detection they are difficult to compare. The encoded data is only a small factor in the latency.

14

Having no prior experience with, or knowledge of Aruco codes (QED :smiley: ) I will take your word for it. It's odd though the site I used allows trackers to be created that cannot be tracked, but that explain the 99 bit. It is of course not your problem because you did not make that site :wink:

However, I did find very little differences in tracking performance between my simple low-payload QR codes and Aruco 5x5_50 codes. Maybe the Aruco codes are a wee bit faster. I did have the feeling upgrading from OS 1.8.0 to 1.9.0 contributed to a faster overal tracking performance, but that may just be my imagination