Streaming Video from Magic Leap 2 to Unity via WebRTC: Video Freezes After First Frame

Unity Development
1

Give us as much detail as possible regarding the issue you're experiencing.

Unity Editor version: 2022.3.5
ML2 OS version:
MLSDK version: 1.9.0
Host OS: (Windows/MacOS) Windows 11

Hello Magic Leap community,

I'm currently working on a project where I'm trying to stream video from my Magic Leap 2 device to a Unity PC application using WebRTC. I've successfully established a connection, and the audio streams perfectly from the Magic Leap device. However, I'm facing an issue with the video stream.

Problem: Once the connection is made, the video on the Unity side only displays the first frame and then freezes. The audio continues to stream without any issues.

Setup:

  • Magic Leap 2: I'm using the default settings and, as far as I know, I can't change or select the codec when sending video from this side.
  • Unity PC App: I'm utilizing the Unity WebRTC package. This package supports:
    • Hardware decoding of video in H264 format.
    • Software decoding of video in VP8 and VP9 formats.

Given that I can't select the codec on the Magic Leap side, I suspect there might be a compatibility issue with the video codec being used by Magic Leap 2 and the decoding capabilities of the Unity WebRTC package.

Request for Help:

  1. Has anyone faced a similar issue when streaming video from Magic Leap to another platform via WebRTC?
  2. Is there a way to change or select the video codec on the Magic Leap side, even if it's not directly exposed in the settings?
  3. Are there any recommended settings or workarounds to ensure smooth video streaming between Magic Leap 2 and Unity using WebRTC?

Any insights, suggestions, or solutions would be greatly appreciated. Thank you in advance for your assistance!

2

Hi @feel_digital,

Welcome to the Magic Leap 2 Developer Forums. We are so grateful to have you here engaging with us.

Are you receiving any error messages after the video freezes?

Best,

El

3

No errors. we see in unity webrtc panel we receiving data. seem like this is Nvidia decoder problem, because solution working on mac with same plugin in unity.