Unity WebRTC: Encoder Queue Full

WebRTC
1

Give us as much detail as possible regarding the issue you’re experiencing.

Unity Editor version: 6000.0.62f1
ML2 OS version: Version 1.12.1
Build B3E.251015.01
Android API Level 29
MLSDK version: 2.6.0-pre.R15

Error messages from logs:

We had put a pause on our Magic Leap unity build for the past year. We recently got back into it and making sure the build is up to date with our new features. We are currently having issues with the calls, specifically the video which stops sending frames after about 5-10 seconds.

Our calls using Unity WebRTC used to work very well. We did update a lot of things when booting the project back up: Unity 6, WebRTC 3.0.0 and of course the ML OS and SDK.

Looking through the android logs im seeing a new system that I dont remember seeing back in the day (MLMRCamera3Hal). Im getting constant chatter from this system for every frame:

2026-02-23 18:43:32.796 3360 3523 Warn Camera3-Device No cvip timestamp available
2026-02-23 18:43:32.796 3294 9174 Info MLMRCamera3Hal ENTRY : processCaptureRequest
2026-02-23 18:43:32.796 3294 9174 Info MLMRCamera3Hal EXIT : processCaptureRequest

From our web rtc stats I consistently get around ~300 encoded frames before the Encoding Queue fills up:

WebRTC stats: {“t”:“2026-02-23T19:29:17.285559Z”,“lp”:“n1”,“rp”:“nl”,“a”:{“snd”:{“lvl”:0.0,“b”:24754},“rcv”:{“lvl”:0.0,“b”:31414}},“v”:{“snd”:{“enc”:291,“df”:290,“kf”:31,“b”:677510}}}

Then the new spam of logs are:

2026-02-23 19:29:17.450 3360 3533 Warn Camera3-Device No cvip timestamp available
2026-02-23 19:29:17.450 3294 3294 Info MLMRCamera3Hal ENTRY : processCaptureRequest
2026-02-23 19:29:17.450 3294 3294 Info MLMRCamera3Hal EXIT : processCaptureRequest
2026-02-23 19:29:17.456 23070 4963 Error org.webrtc.Logging HardwareVideoEncoder: Dropped frame, encoder queue full
2026-02-23 19:29:17.472 23070 4963 Error org.webrtc.Logging HardwareVideoEncoder: Dropped frame, encoder queue full

Some initial Encoder logs if it helps:

org.webrtc.Logging HardwareVideoEncoder: Format: {color-format=21, i-frame-interval=3600, mime=video/avc, width=960, bitrate-mode=2, bitrate=283000, frame-rate=60.0, height=720}

And right after it updates the format, unsure if this is expected:

org.webrtc.Logging HardwareVideoEncoder: updateInputFormat format: {color-transfer=0, color-format=21, slice-height=720, image-data=java.nio.HeapByteBuffer[pos=0 lim=104 cap=104], mime=video/raw, width=960, stride=960, color-range=0, color-standard=0, height=720} stride: 960 sliceHeight: 720 isSemiPlanar: true frameSizeBytes: 1036800

Looked at your forums and I heard mention of some Unity WebRTC examples but going through your examples I dont see anything relating to WebRTC.

Curious if there is a new way of accessing the camera that I should be doing on the Magic Leap. We have the same Android webrtc stack working on the Argo and Realwear but I’m currently struggling with this one.

Any info would be greatly appreciated!

2

Found your WebRTC repo. You guys are aiming for:
private const int WIDTH = 1440;
private const int HEIGHT = 1080;

webCameraName = “Camera 2”;

We are currently aiming for lower than that and using the same camera (Which is the only MR to my knowledge):

Debug ThirdEye-HAL Setting up recording session Width 960 Height 720 Format 35 usage 515 flags 0 Type 3 FPS 30

But the Buffer is allocating a much bigger chunk:
Debug MLMRCamera-Buffers init BufferItemConsumer setting width 2048 height 1536 buffer count 6 format 35

Not sure if that’s intended or not, I’m guessing you guys just downscale and always allocate the max? Im not usually the guy that works on our calls but the others are busy. This is a bit of a new area for me.

I’ve tried VP8 and that seems to fill up even faster than the H264.

Also, from the following log line, I dont think the device should be defaulting to the mesa encoder:
Info org.webrtc.Logging HardwareVideoEncoder: initEncode name: OMX.mesa.video_encoder.avc type: H264 width: 960 height: 720 framerate_fps: 60 bitrate_kbps: 283 surface mode: false

3

Hey @nathan ,

Correct, it should not be defaulting to the mesa encoder, to me that seems like the main issue here.
This may be coming from the Unity WebRTC Package.

If possible, please try and force Unity WebRTC to initialize with the hardware encoder:

WebRTC.Initialize(EncoderType.Hardware);

Please let me know if this helps at all!

Best,
Corey

4

They seemed to have removed that call in 2022. com.unity.webrtc/CHANGELOG.md at main · Unity-Technologies/com.unity.webrtc · GitHub

Do you recommend using an older WebRTC package?

5

I just got VP8 to work, I have no idea what I did earlier this week when it didn’t. Currently the feed is very laggy though, probably getting ~10 fps.

Has there been someone who managed to get the Hardware accelerated H264 encoder to work with Unity WebRTC?

6

@nathan ,

I’ve been looking into this a bit more for you.
You can try using an older WebRTC package if that’s viable for your project to see if you can force the encoder to be hardware driven.

Also, just out of curiosity, could you try an older version of Unity, like 2022.3 LTS to see if you get the same behavior?

If not, that’s totally fine, just trying to pin down the root cause here.

Best,
Corey