Using the Unity WebRTC Plugin together with the MagicLeap SDK

We are currently working on an app for which we require features from the MagicLeap SDK. We also need WebRTC, and had issues with the MLWebRTC integration, e.g. when turning the camera on and off, as well as supporting multiple codecs.
We thus decided to use Unity WebRTC, which offers the additional benefit to be platform independent. In current PRs on the Unity WebRTC Github project, support for Android x86 is tested, especially due to interest and contributions from other MagicLeap developers. ([REQUEST]: Android x86_64 support · Issue #802 · Unity-Technologies/com.unity.webrtc · GitHub)

Sadly, we do not seem to be able to integrate both Unity WebRTC and the MagicLeap SDK in our project at the same time. While our app starts fine if only one of the two plugins is present, it fails to load (just a black screen, no logs, no relevant error messages) if both are present.

We assume that, as both Unity WebRTC and the MagicLeap SDK integrate WebRTC into the project, there must be a conflict with e.g. duplicate WebRTC .aar files, or libraries in the .so format. As we could not find a way to remove WebRTC from the MagicLeap SDK to confirm if that is the root of our issue, we decided to ask for support in this forum. If there is any way to exclude WebRTC from the MagicLeap SDK, that would help us a lot with development.

Unity Editor version: 2022.3.0f1
ML2 OS version: May 2023
MLSDK version: 1.3.0-dev1
Host OS: (Windows/MacOS) Windows 11

Error messages from logs (syntax-highlighting is supported via Markdown): None, sadly.

Hi @crucial-sailor,

Welcome to the Magic Leap 2 developer forums. We are so grateful to have you here interacting with us.

I have reached out to our team to see if it is possible to exclude WebRTC from the SDK when importing to unity.

I would also like to ask, in more detail, what were the issues you were encountering when using the MLWebRTC integration? This feedback will help tremendously when improving these packages.



Try this pre-built plugin for Unity's WebRTC library: (25.0 MB)

To use it, replace the default .aar file with the one included here. (default location: com.unity.webrtc/runtime/Plugins/Android/)

It would also be helpful to get some logs from your device when starting the application.

Hi @etucker,

I did some more testing on our side and will try to format this response in two parts, one regarding Unity WebRTC + ML SDK, the other regarding MLWebRTC.

Unity WebRTC + ML SDK

Regarding the Google Drive Link

Thank you very much for offering to use the .aar built by you – we already have built and tested a version as well (for Unity WebRTC), and it should work:
We created a minimal OpenXR app without the ML SDK for the Magic Leap. There, we created a WebRTC peer and a local offer, and printed the resulting SDP. The whole process works fine.

Regarding Logs

We updated our Unity SDK to 1.8.0, and switched to version 1.3.0-dev2, just to use the latest software. The Magic Leap is still on the May 2023 version though.

As long as either the MLSDK or the .aar is part of the project, the app will not load. There is no Unity log message at all. Logcat does not give useful warnings or errors either.

Most likely while loading the app libraries, the app gets stuck (after 600ms) and does not even get far enough to start any Unity logging process.

The only logs that don’t seem meaningless are:

  • 2023/07/19 09:45:21.241 6545 6545 Error […] Not starting debugger since process cannot load the jdwp agent.
  • 2023/07/19 09:45:21.818 6545 6566 Error libprocessgroup set_timerslack_ns write failed: Operation not permitted
  • 2023/07/19 09:45:21.879 6545 6566 Debug vulkan searching for layers in '/data/app/[…]==/lib/x86_64'
  • 2023/07/19 09:45:21.879 6545 6566 Debug vulkan searching for layers in '/data/app/[…]==/base.apk!/lib/x86_64'
  • 2023/07/19 09:45:21.898 6545 6545 Warn UnityMain type=1400 audit(0.0:259): avc: denied { search } for name="traces" dev="nvme0n1p37" ino=278531 scontext=u:r:untrusted_app:s0:c112,c256,c512,c768 tcontext=u:object_r:trace_data_file:s0 tclass=dir permissive=0 (shows 2x in a row)
  • 2023/07/19 09:45:21.907 6545 6566 Info EGL-MAIN found extension DRI2_Blob version 1 (this is where the logs end)

As the app does not crash, we need to close it manually on the device. As said before, without either the .aar or the ML SDK the app actually starts, and shows our initial scene.

Did it work for you?

Can you observe a similar behavior in an app where both the Unity WebRTC SDK with the modified .aar for x86 and the ML SDK are present?

Regarding MLWebRTC

The WebRTC sample app

There is a WebRTC sample app, as documented in I have a few issues to note about that app.

  • For Python3, getargspec in needs to be replace by getfullargspec, furthermore, while works, does not work. That took a few minutes to fix, so I guess it would be helpful to write a hint about that in the developer guide.
  • The app seems to work fine as long as the MagicLeap does not go into sleep mode during a call. This can already happen when the user looks elsewhere than through the glasses themselves. As soon as that happens, when the app wakes up from sleep again, the camera access is broken (if we’re lucky) or the whole system is broken (tracking is permanently lost on top of the broken camera, and we need to restart the device). Logs are shown below.

When the camera is “broken” that also means that the green recording indicator never disappears, so the camera access is not cleaned up properly, which also indicates potential memory leaks.

Can we disable the “sleep” behavior of the Magic Leap? It would be quite unfortunate if the app goes to sleep every time the user looks elsewhere than through the glasses. Of course, going to sleep when the device is not on somebody’s head makes sense, but, coming from the HoloLens, it would be quite nice if the device would stay awake no matter what the eye tracking system is measuring.

Using MLWebRTC in our app

Compared to native and Unity WebRTC, we noticed that some interfaces were slightly changed (e.g. for setting ICE servers during peer creation), while others are completely missing.
We don’t need all configurations, but the following issues remain:

  • Codec support: We need to support more than just H264. Unity WebRTC, for example, supports VP8 as well. Software encoding would be sufficient here.
  • As with the sample app, as soon as the app goes to sleep, we run into issues regarding the camera and microphone access.
  • Destruction/teardown and the whole setup was not clear to us, due to lacking documentation. It took some trial and error to not leave the camera open when closing a WebRTC call, and we are still not sure if there are potential memory leaks depending on the order in which the camera and tracks are closed.

Hi @crucial-sailor ,

Regarding Unity WebRTC Package Compatibility

We are investigating this issue. I was able to reproduce the application not starting properly after replacing the Android Plugin. I also observed the behavior describe regarding the Magic Leap SDK and WebRTC plugin conflict. The application would start correctly when compiled as a standard 2D application but not when the Magic Leap SDK was present. We are working quickly to come up with a resolution.

Sorry for the inconvenience.

Regarding the ML WebRTC Documentation/ Instructions.

Thank you for letting us know about the issue regarding the demo/instructions. I passed this information along to the appropriate team, we will get that page updated shortly.

Regarding Sleep:

Thank you for providing the logs and reproduction steps. I submitted this information to our voice of customer team. We are tracking a few issues regarding sleep mode and are working on improving it's functionality.

You can disable sleep mode inside your device's Battery Settings in the Settings application.

Note: that sleep was enabled by default on the 1.3.0-dev2 OS, but this change will be reverted in the 1.3.0 OS release coming next month.

Regarding VP8 Encoding

We are investigating this and will get back to your shortly. Note, Magic Leap 2 does not support VP8 hardware encoding. Can you share more information about why you prefer using VP8 instead of H264?

Hi @kbabilinski,

thank you very much for your feedback and time. We are looking forward to the resolution of this compatibility issue.

Regarding VP8: I can't give any details, but the main argument is device and browser compatibility. We require support for VP8 (and optionally VP9).

I submitted this feedback to our voice of customer team. We are also investigating the issue with the com.unity.webrtc package preventing apps from starting properly.

Hi @kbabilinski,

we just wanted to check and ask if there is already some progress regarding the conflicting libraries, also to make sure that this thread does not become inactive. We are still looking forward to a resolution of this issue, and will be happy to port and test our app on the MagicLeap afterwards.

We have been able to reproduce this issue and our engineering team is investigating this issue. We will make sure to highlight the fix in future release notes.

Hello @kbabilinski,

Thank you for taking your time to reply to this thread, I was just curious if there's an update on this as a project I'm working on relies heavily on Unity WebRTC to work. I have a custom Unity WebRTC package that works on the Magic Leap 2 but having the Magic Leap SDK breaks the application. Just curious if a fix is coming soon or if there's a work around as having both packages would be the best outcome for the project. Thank you :slight_smile:

We are working on resolving this issue, once the issue is resolved we will post an update on the forum to let everyone know.


we are still encountering some issues with the Unity WebRTC library and would like to ask if you have encountered similar problems.

  • We cannot use H264. For some reason, the SDP of our publishing peer does not even offer H264.
  • The VP8 performance is really bad. Much worse than we expected with software encoding on this chipset, given that we have a better experience with older phones and as any laptop with a similar processor (without hardware encoding) should also be able to provide at least 30FPS at 720p resolution. We only get about 15FPS at 720p. It would be great if you could try out VP8 as well and tell us about your experience.

We will also contact Unity via Git, but their patches do not indicate any changes to the original codebase that could lead to worse performance, unless their downgrade from Java 11 to Java 8 can cause such issues. Also, the performance of Unity WebRTC on desktop is much better, even though it also only uses software encoding.

We are thus wondering if the Android compilation path of WebRTC assumes / favors ARM-based processors and could lead to issues when using an x64 chip instead? Or maybe we need to change some compilation flags to make sure that performance is acceptable?

We also want to share some additional information from debugging:

  • According to CPU statistics from the Unity Editor, the MagicLeap 2 can keep constant 60FPS for the user.
  • When publishing a single video stream, CPU usage (according to Android Studio) goes up from 11% to 28%. While this is a huge increase, there still seems to be a lot of unused CPU time.
  • On the Meta Quest Pro, we see a similar increase, but video FPS are much higher. The chipset of the Meta Quest Pro should not be more powerful than the one used in the Magic Leap 2, according to our short research.

We get the following results when we compare different platforms, when taking a look at the average frame encoding time in ms, with VP8, software encoding, FullHD:

  • Meta Quest Pro (Android, ARM64): 40ms
  • MagicLeap 2 (Android, x64): 159ms
  • Unity Desktop (Windows, x64): 14ms

The performance on the Meta Quest Pro is very unstable though, the worst encode time was 80ms. Desktop and ML2 are both very stable regarding their encode times, no matter how much the user is moving their head.

Thank you for providing the detailed debug information. I have reported your feedback. I will let you know our findings.

Any news regarding the UnityWebRTC package compatibility ?

Our next software release should resolve the issue with Application's failing to load when the Unity WebRTC Package is present. We aim to release the update before the end of October. However, you will continue needing to build the plugin with x86_64 support until Unity releases an update that is compiled with the new build script.

1 Like