We can log a bug internally. In the meantime, I should be able tweak the plugin code a bit to ensure that it generates a crash dump when this occurs and we can work with the team to resolve the problem.
In the meantime, here's a quick hack you can try. In your Plugins folder \MagicLeapMarkerTracking\Source\MagicLeapMarkerTracking\Private\MagicLeapMarkerTrackingModule.cpp
just before the call to xrCreateMarkerDetectorML, try adding a call to DetectorDataByDetector.Reset();
I have updated the module.cpp and added that line. How do I force it recompile the plugin source when relaunching project? It seems to just launch as reg without recompiling…
Also adding this code, what change am I going to see?
Thanks! With regard to rebuilding the plugin, yea I noticed that too. There might be a better way to force it to recompile, but if you close the UE editor and just delete the binaries, then it'll prompt you to recompile the plugin the next time that you open the project. In the marker tracking plugin folder, you can delete the Binaries, Content, and Intermediate folder, and it'll regenerate them when you reopen the project-
That change is intended to [hopefully] rule out any issue with improper cleanup when running in the editor. (the elements in that map don't get cleared when you start/stop in the editor) [As a side-effect, it will also make it so that you can only have a single marker tracker component instance, but there's probably not much practical reason to have more than one.] Likely you won't see a difference in behavior, but that map contains handles to the tracker as output by OpenXR apis, so you could see errors like you previously posted if that was the issue. It seemed easy enough to try though.
Just to be clear, you have it working on your end with UE 5.4.4 commercial and reading aruco with remote rendering?
The only thing that works, is apk on the headset. How can we test if its urneal, our enviornemnt, or remote rendering causing the issue? Just curious do you have steam VR installed?
LogDebuggerCommands: Repeating last play command: VR Preview
LogPlayLevel: PlayLevel: No blueprints needed recompiling
LogOnline: OSS: Created online subsystem instance for: NULL
LogOnline: OSS: TryLoadSubsystemAndSetDefault: Loaded subsystem for type [NULL]
LogPlayLevel: Creating play world package: /Game/UEDPIE_0_MarkerTest
LogPlayLevel: PIE: StaticDuplicateObject took: (0.001718s)
LogPlayLevel: PIE: Created PIE world by copying editor world from /Game/MarkerTest.MarkerTest to /Game/UEDPIE_0_MarkerTest.MarkerTest (0.001762s)
LogUObjectHash: Compacting FUObjectHashTables data took 0.48ms
LogWorldMetrics: [UWorldMetricsSubsystem::Initialize]
LogPlayLevel: PIE: World Init took: (0.001042s)
LogAudio: Display: Creating Audio Device: Id: 2, Scope: Unique, Realtime: True
LogAudioMixer: Display: Audio Mixer Platform Settings:
LogAudioMixer: Display: Sample Rate: 48000
LogAudioMixer: Display: Callback Buffer Frame Size Requested: 1024
LogAudioMixer: Display: Callback Buffer Frame Size To Use: 1024
LogAudioMixer: Display: Number of buffers to queue: 1
LogAudioMixer: Display: Max Channels (voices): 32
LogAudioMixer: Display: Number of Async Source Workers: 4
LogAudio: Display: AudioDevice MaxSources: 32
LogAudio: Display: Audio Spatialization Plugin: None (built-in).
LogAudio: Display: Audio Reverb Plugin: None (built-in).
LogAudio: Display: Audio Occlusion Plugin: None (built-in).
LogAudioMixer: Display: Initializing audio mixer using platform API: 'XAudio2'
LogAudioMixer: Display: Using Audio Hardware Device Speakers (Realtek(R) Audio)
LogAudioMixer: Display: Initializing Sound Submixes...
LogAudioMixer: Display: Creating Master Submix 'MasterSubmixDefault'
LogAudioMixer: Display: Creating Master Submix 'MasterReverbSubmixDefault'
LogAudioMixer: FMixerPlatformXAudio2::StartAudioStream() called. InstanceID=2
LogAudioMixer: Display: Output buffers initialized: Frames=1024, Channels=2, Samples=2048, InstanceID=2
LogAudioMixer: Display: Starting AudioMixerPlatformInterface::RunInternal(), InstanceID=2
LogAudioMixer: Display: FMixerPlatformXAudio2::SubmitBuffer() called for the first time. InstanceID=2
LogInit: FAudioDevice initialized with ID 2.
LogAudio: Display: Audio Device (ID: 2) registered with world 'MarkerTest'.
LogAudioMixer: Initializing Audio Bus Subsystem for audio device with ID 2
LogSlate: Updating window title bar state: overlay mode, drag disabled, window buttons hidden, title bar hidden
LogLoad: Game class is 'XrGameMode_C'
LogWorld: Bringing World /Game/UEDPIE_0_MarkerTest.MarkerTest up for play (max tick rate 0) at 2024.09.23-16.08.08
LogWorld: Bringing up level for play took: 0.000760
LogOnline: OSS: Created online subsystem instance for: :Context_2
PIE: Server logged in
PIE: Play in editor total start time 0.166 seconds.
LogHMD: Warning: No UPlayerMappableInputConfig provided in the OpenXR Input project settings, action bindings will not be visible to the OpenXR runtime. Action/Axis mappings from the Input configuration are deprecated for XR in favor of Enhanced Input.
LogHMD: Warning: Requesting 10 bit swapchain, but not supported: fall back to 8bpc
LogRenderer: Warning: Resizing VR buffer to 2512 by 1532
LogSlate: Took 0.000406 seconds to synchronously load lazily loaded font '../../../Engine/Content/Slate/Fonts/Roboto-Regular.ttf' (155K)
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogProfilingDebugging: Allocated a 1024 x 1024 texture for HMD canvas layer
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogBlueprintUserMessages: [MLMarkerTracking_Blueprint_C_0] X=0.000 Y=0.000 Z=0.000
LogSlate: Updating window title bar state: overlay mode, drag disabled, window buttons hidden, title bar hidden
LogWorld: BeginTearingDown for /Game/UEDPIE_0_MarkerTest
LogSlate: Window 'MLRRMarkerTest Preview [NetMode: Standalone 0] (64-bit/PC D3D SM6) OpenXR '1.15.131' (1.15.131)' being destroyed
LogWorld: UWorld::CleanupWorld for MarkerTest, bSessionEnded=true, bCleanupResources=true
LogSlate: InvalidateAllWidgets triggered. All widgets were invalidated
LogWorldMetrics: [UWorldMetricsSubsystem::Deinitialize]
LogWorldMetrics: [UWorldMetricsSubsystem::Clear]
LogPlayLevel: Display: Shutting down PIE online subsystems
LogSlate: InvalidateAllWidgets triggered. All widgets were invalidated
LogAudioMixer: Deinitializing Audio Bus Subsystem for audio device with ID 2
LogAudioMixer: FMixerPlatformXAudio2::StopAudioStream() called. InstanceID=2
LogAudioMixer: FMixerPlatformXAudio2::StopAudioStream() called. InstanceID=2
LogUObjectHash: Compacting FUObjectHashTables data took 0.56ms
LogPlayLevel: Display: Destroying online subsystem :Context_2
LogOutputDevice: Warning: Script Stack (0 frames) :
LogStats: FPlatformStackWalk::StackWalkAndDump - 0.051 s
LogOutputDevice: Error: === Handled ensure: ===
LogOutputDevice: Error: Ensure condition failed: ((Result) >= 0) [File:D:\build\++UE5\Sync\Engine\Plugins\Runtime\OpenXR\Source\OpenXRHMD\Private\OpenXRHMD.cpp] [Line: 4218]
LogOutputDevice: Error: OpenXR call failed with result XR_ERROR_HANDLE_INVALID
LogOutputDevice: Error: Stack:
LogOutputDevice: Error: [Callstack] 0x00007ff8c20680a6 UnrealEditor-OpenXRHMD.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff8c208837d UnrealEditor-OpenXRHMD.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff8c2089d4d UnrealEditor-OpenXRHMD.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff8c20678cd UnrealEditor-OpenXRHMD.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff8c2082568 UnrealEditor-OpenXRHMD.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff8d68bb641 UnrealEditor-Engine.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff8d381c04c UnrealEditor-UnrealEd.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff8d449f6b6 UnrealEditor-UnrealEd.dll!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff657758e0b UnrealEditor.exe!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff65777e33c UnrealEditor.exe!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff65777e42a UnrealEditor.exe!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff6577818a4 UnrealEditor.exe!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff6577970c4 UnrealEditor.exe!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff65779a37a UnrealEditor.exe!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff9df9b7374 KERNEL32.DLL!UnknownFunction []
LogOutputDevice: Error: [Callstack] 0x00007ff9e01bcc91 ntdll.dll!UnknownFunction []
LogStats: SubmitErrorReport - 0.000 s
LogStats: SendNewReport - 1.864 s
LogStats: FDebug::EnsureFailed - 1.924 s
LogDerivedDataCache: C:/Users/b1216061/AppData/Local/UnrealEngine/Common/DerivedDataCache: Maintenance finished in +00:00:00.073 and deleted 0 files with total size 0 MiB and 0 empty folders. Scanned 198 files in 401 folders with total size 64 MiB.
Yes, it does work for me. I'm using these component versions-
UE v5.4.4
ML Remote Rendering service and viewer app v1.15.131
ML OS v1.9.0
ML Unreal plugins v1.6.0
I don't have SteamVR installed on this machine, but that's a good question as if you were using multiple OpenXR runtimes/devices on the same machine, then it would cause a problem if a non-MagicLeap runtime were set active. If you're able to use remote rendering with Unreal at all though, (you're at least able to connect to the ML2 and see content) then the active OpenXR runtime must be set correctly. If you want to sanity check that ML remote render is active, you can check the value in the Windows registry. To do that, open the Windows reg editor (Start -> type 'regedit'). Then, navigate to HKLM\SOFTWARE\Khronos\OpenXR\1 and make sure that the ActiveRuntime value is set to the path to openxr_windrunner.json.
Thanks for the log. It looks like that error occurred when the session was being torn down (like when you hit stop). It may or may not be directly related. The underlying tracker apis do leverage OpenXR spaces to locate markers, but that wouldn't indicate that the system was not able to locate the marker at runtime.
Interestingly, if that's the complete log, then it does appear as though you no longer are seeing errors when these functions are called. Is that true?
LogMagicLeapMarkerTracking: Error: Tick: xrGetMarkerDetectorStateML failure XR_ERROR_HANDLE_INVALID
LogMagicLeapMarkerTracking: Error: Tick: xrSnapshotMarkerDetectorML failure XR_ERROR_HANDLE_INVALID
Btw, we logged a bug on this internally, although hopefully we can find a resolution to unblock you here.
It can be a little fiddly, but to eliminate your network as a concern, you can try using remote render over a usb connection. This is possible because the network connection sharing feature in Android basically emulates a usb network adapter. We have some documentation on how to set it up here if you want to give it a try-
Btw, that guide page suggests using an adb command to enable it, but you can also just use the Settings gui on the device. In Settings -> Network & Internet -> Hotspot & tethering -> enable "USB tethering". That does the same thing as the rndis command that the page suggests.
It may also be interesting to take a look at logs from ML remote rendering. We have a guide here on how to collect them-
If you do run that, and if you would be able to send over the logs, I'd be happy to take a look (feel free to dm it to me). If you do collect them, the guide page suggests running the FetchLogs.ps1 script with the -HostMetrics flag. Please also include -HostLogs so we get behavioral logs as well. If you have your ML2 connected via adb, it you run the script with -AllLogs instead, then it will also include logs from the remote viewer app which could also be interesting.
Failing all of that, we may have to resort to just directly debugging the plugin code. There are a couple of ways we could go about that.
Thanks for following up. So my Reg Edit looks the same as yours does. AS far as the UE specific logs I have noticed the logs report differently based upon initial play and second time I hit play in the editor. I have included those both in my log package I sent you in direct email.
I have tried the enabling Usb tethering form the device itself and hooked it up with usb cable directly to pc. No change.
For sanity check can you verify and post your environment variables you have setup, I want to cross reference. Would it be possible to make apk of your working Unreal Proj so we can put it on the headset physically and see if that works?
Also if you hook up print string to this break… are you getting active numbers from your aruco code besides 0,0,0?
Thanks for sending over the logs! I took a quick look through them. It looks like you reproduced the issue that I mentioned where the tracker handles are not properly cleaned up when you start/stop in the UE editor (errors appear on the second run). However, that doesn't explain why the position is always 0. It really is a bit odd since it appears as though the system does detect the marker. The pose is just wrong.
Yes, I get a reasonable pose (position and orientation) for detected markers. One known caveat is that if you rely on the marker size estimation (which happens if you enter a negative value for the marker size), then it takes a few seconds for the tracker to estimate it and start tracking. It looks like you've been setting the size to 10cm though, so that's not the issue here.
For now, maybe we can both try working through some additional tests. I'll follow up privately with details on how we could go about that.