Magic Leap Spectator App issues with OpenXR and MRTK3

I am currently attempting to make my OpenXR simulation with MRTK3 interactivity compatible with Magic Leap's Spectator App on iOS.

Currently, when attempting to use the spectator app with my sim, the sim builds and loads fine. It performs and functions as expected when not attempting to synch with the spectator app. When I do attempt to use the spectator app itself, I can find my device on the network, connect to it, and scan the ARUco/marker with the app on the phone. However, when attempting the next step in the pairing process, to scan the ARUco/marker with my headset, the headset is unable to process the ARUco. Breakpoint debugging shows that the sim isn't even attempting to process the marker, though scanning is enabled.

I want to make it clear that the MarkerUnderstanding tracking works completely fine with this setup. My QR codes that are used for my code's logic are recognized and properly used. It is just the marker tracking that comes from the Spectator App that does not work.

From previous threads, I know that the Magic Leap Spectator app is reliant on Magic Leap's XR plugin (com.unity.xr.magicleap), however, whenever I install and enable this plugin, my app crashes on startup, although I receive no build errors, and can enter play mode just fine in my Unity editor.

Is Spectator App incompatible with OpenXR and MRTK3? If not what could be wrong with my set up? If it is, are there any plans soon to fix compatibility issues?

Unity Editor version: 2022.3.31f1
ML2 OS version: 1.7.0 Secure
Unity SDK version: 2.2.0
Magic Leap Spectator Version: 1.3.3
Magic Leap MRTK3 Version: 1.0.0
OpenXR Plugin: 1.10.0
Mixed Reality OpenXR Plugin Version: 1.10.1
Host OS: Windows

Bumping this for visibility.

Hi @tyler.morris
I am looking into this, sorry for the delay. I will get back to you asap
-Sidney

1 Like

Hi I spoke to some of the engineers and they said they were aware of an issue with mrtk, openxr, and spectator, but that it should have been cleared up by the latest versions. Could you try updating versions and let me know if that fixes it?

Thanks for getting back to me. I have made sure my magic leap, openxr, and MRTK versions are all up to date.

I just tested with these updated packages and still have the above issues where I crash no matter what I try.

Thank you for trying that. I will let them know and try to track down some more information on this.

Ok, I got some further guidance. Could you try enabling Perception snapshots and see if that gets it working? if it does then there are probably some URP settings to check as well

Still receiving crashes with this enabled. To confirm, I also still should have the Magix Leap XR plugin enabled for this setup, correct?

Actually I was incorrect. Disable and remove the old plugin (Magic leap xr for unity). Additionally it was brought to my attention that there is a bug we are working with Unity to fix causing crashes with the latest "Magic Leap XR Plugin for Unity" version 7.1.0, that may be the root cause of the crashing problem here. You can check via the changelog

That did it! I only had the XR plugin installed for spectator, the app worked fine without it but the spectator functionality didn't work. Looks like enabling Perception Snapshots allowed the spectator to function.

However, this seems to conflict with my MarkerUnderstanding Functionality. The two seem to be mutually exclusive where if I scan for the spectator app, I can no longer scan my QR Code for the rest of my app's functionality. If I scan the QR Code, I cannot scan for spectator app, and also my QR Code position seems to be significantly warped rotation-wise, and I am unable to execute any of my rescan/recalibration functions. Any ideas for these issues? Glad we're making progress here.

There seem to be some issues with a few minor versions of unity. Others on the team reported issues with versions past 2022.3.11f1 with version 2022.3.33f1 fixing the issue. Could you try upgrading to version 2022.3.33f1 and see if that fixes the problem?

Looks like we're getting closer. Spectator seems to work fine in tandem with vertical QR Codes. However, my horizontal QR codes don't seem to function at all.

That is really interesting. I will ask if anyone has seen this. Just for clarity, are you saying that rotating the qr code 90 degrees makes it stop working?

Rotating in 3d space yes, so having a QR code sit on a stand or taped to a wall works. Having one flat against a table or counter top does not.