I am currently developing an application using the Magic Leap headset and I've encountered an issue related to disconnecting and then reestablishing WebRTC call sessions.
Unity Editor version: 2022-2.0f1
ML2 OS version: 1.3.0-dev2
MLSDK version: 1.7.0
Host OS: Windows 11
Here's the disconnect function I'm currently using in my code:
public void Disconnect()
{
if (connection == null)
{
return;
}
webRequestManager.HttpPost(serverURI + "/logout/" + localId, string.Empty);
if (dataChannel != null)
{
dataChannel.OnClosed -= OnDataChannelClosed;
dataChannel.OnOpened -= OnDataChannelOpened;
dataChannel.OnMessageText -= OnDataChannelTextMessage;
dataChannel.OnMessageBinary -= OnDataChannelBinaryMessage;
dataChannel = null;
}
UnsubscribeFromConnection(connection);
connection.Destroy();
connection = null;
localVideoSource.DestroyLocal();
remoteMediaStream = null;
waitingForAnswer = false;
waitingForAnswerGetRequest = false;
connectButton.gameObject.SetActive(true);
serverInput.gameObject.SetActive(true);
localVideoSinkBehavior.gameObject.SetActive(false);
remoteStatusText.text = "Disconnected";
//localStatusText.text = "";
remoteVideoSinkBehavior.VideoSink.SetStream(null);
remoteAudioSinkBehavior.gameObject.SetActive(false);
remoteVideoSinkBehavior.gameObject.SetActive(false);
disconnectUI.SetActive(false);
messageUI.SetActive(false);
//dataChannelText.text = "";
localVideoSourceDropdown.interactable = true;
localVideoSizeDropdownRGB.interactable = true;
localVideoSizeDropdownMR.interactable = true;
remoteId = "";
localId = "";
}
I am getting the following crash, when I am trying to reopen webrtc calling screen to restart the calling functionality in single session of the app.
2023/07/21 16:39:15.754 7534 7567 Info Unity ICE Servers: UnityEngine.XR.MagicLeap.MLWebRTC+IceServer[]
2023/07/21 16:39:15.754 7534 7567 Info Unity UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
2023/07/21 16:39:15.754 7534 7567 Info Unity UnityEngine.Logger:Log(LogType, Object)
2023/07/21 16:39:15.754 7534 7567 Info Unity UnityEngine.Debug:Log(Object)
2023/07/21 16:39:15.754 7534 7567 Info Unity MagicLeap.Examples.<WaitForIceServers>d__100:MoveNext() (at D:\Android\AR\wavefunction\ConceptHealth-A-Consult\Assets\ExpertCall\Scripts\WebRTC\MLWebRTCExample.cs:918)
2023/07/21 16:39:15.754 7534 7567 Info Unity UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr) (at \home\bokken\build\output\unity\unity\Runtime\Export\Scripting\Coroutines.cs:17)
2023/07/21 16:39:15.754 7534 7567 Info Unity UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator) (at \home\bokken\build\output\unity\unity\Runtime\Export\Scripting\MonoBehaviour.bindings.cs:112)
2023/07/21 16:39:15.754 7534 7567 Info Unity MagicLeap.Examples.MLWebRTCExample:CreateIceServers() (at D:\Android\AR\wavefunction\ConceptHealth-A-Consult\Assets\ExpertCall\Scripts\WebRTC\MLWebRTCExample.cs:954)
2023/07/21 16:39:15.754 7534 7567 Info Unity MagicLeap.Examples.<<Login>b__67_0>d:MoveNext() (at D:\Android\AR\wavefunction\ConceptHealth-A-Consult\Assets\ExpertCall\Scripts\WebRTC\MLWebRTCExample.cs:325)
2023/07/21 16:39:15.754 7534 7567 Info Unity System.Runtime.CompilerServ
2023/07/21 16:39:15.755 7534 7567 Info luminkarrot nova/frameworks/kali/kaliwebrtc/simple/src/Factory.cpp:71 Creating peer connection
2023/07/21 16:39:15.756 7534 9107 Debug luminkarrot nova/frameworks/kali/kaliwebrtc/videofactory/src/KarrotVideoFactory.cpp:197 VideoEncoderFactory:GetSupportedFormats
2023/07/21 16:39:15.760 7534 7567 Info Unity GET YUV FRAME
2023/07/21 16:39:15.760 7534 7567 Info Unity UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
2023/07/21 16:39:15.760 7534 7567 Info Unity UnityEngine.Logger:Log(LogType, Object)
2023/07/21 16:39:15.760 7534 7567 Info Unity UnityEngine.Debug:Log(Object)
2023/07/21 16:39:15.760 7534 7567 Info Unity MagicLeap.Core.MLWebRTCVideoSinkBehavior:RenderWebRTCFrameYUV(Frame) (at D:\Android\AR\wavefunction\ConceptHealth-A-Consult\Packages\com.magicleap.unitysdk\Runtime\APIs\WebRTC\Scripts\MLWebRTCVideoSinkBehavior.cs:265)
2023/07/21 16:39:15.760 7534 7567 Info Unity MagicLeap.Core.MLWebRTCVideoSinkBehavior:RenderWebRTCFrame(Frame) (at D:\Android\AR\wavefunction\ConceptHealth-A-Consult\Packages\com.magicleap.unitysdk\Runtime\APIs\WebRTC\Scripts\MLWebRTCVideoSinkBehavior.cs:182)
2023/07/21 16:39:15.760 7534 7567 Info Unity MagicLeap.Core.MLWebRTCVideoSinkBehavior:Update() (at D:\Android\AR\wavefunction\ConceptHealth-A-Consult\Packages\com.magicleap.unitysdk\Runtime\APIs\WebRTC\Scripts\MLWebRTCVideoSinkBehavior.cs:131)
2023/07/21 16:39:15.760 7534 7567 Info Unity
2023/07/21 16:39:15.760 7534 17835 Info ml_camera_client Camonly OnUnavailable CamId = 0
2023/07/21 16:39:15.760 7534 7534 Warn UnityGfxDeviceW type=1400 audit(0.0:5361): avc: denied { search } for name="traces" dev="nvme0n1p37" ino=7340035 scontext=u:r:untrusted_app:s0:c105,c256,c512,c768 tcontext=u:object_r:trace_data_file:s0 tclass=dir permissive=0
2023/07/21 16:39:15.772 7534 17828 Info Unity StartCapture with stream config: CaptureType : Video, Width : 1920, Height : 1080, OutputFormat : YUV_420_888, MediaRecorder : NULL
2023/07/21 16:39:15.772 7534 17828 Info Unity UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
2023/07/21 16:39:15.772 7534 17828 Info Unity UnityEngine.Logger:Log(LogType, Object)
2023/07/21 16:39:15.772 7534 17828 Info Unity UnityEngine.Debug:Log(Object)
2023/07/21 16:39:15.772 7534 17828 Info Unity UnityEngine.XR.MagicLeap.MLCameraVideoSource:StartCapture() (at D:\Android\AR\wavefunction\ConceptHealth-A-Consult\Packages\com.magicleap.unitysdk\Runtime\APIs\WebRTC\API\MLWebRTCCameraVideoSource.cs:189)
2023/07/21 16:39:15.772 7534 17828 Info Unity UnityEngine.XR.MagicLeap.MLCameraVideoSource:CreateLocal(MLCameraBase, CaptureConfig, MLResult&, String, Renderer, Boolean, Boolean) (at D:\Android\AR\wavefunction\ConceptHealth-A-Consult\Packages\com.magicleap.unitysdk\Runtime\APIs\WebRTC\API\MLWebRTCCameraVideoSource.cs:112)
2023/07/21 16:39:15.772 7534 17828 Info Unity MagicLeap.Examples.<>c__DisplayClass68_0:<CreateLocalMediaStream>b__0() (at D:\Android\AR\wavefunction\ConceptHealth-A-Consult\Assets\ExpertCall\Scripts\WebRTC\MLWebRTCExample.cs:456)
2023/07/21 16:39:15.772 7534 17828 Info Unity System.Threading.Tasks.Task`1:InnerInvoke()
2023/07/21 16:39:15.772 7534 17828 Info Unity System.Threading.Tasks.Task:Execute()
2023/07/21 16:39:15.772 7534 17828 Info ml_camera_client prepareCapture captureFrameRate = 30fps numStreamConfigs = 1
2023/07/21 16:39:15.772 7534 17828 Info ml_camera_client Stream 0 type = Video resolution = 1080P
2023/07/21 16:39:15.772 7534 17828 Info ml_camera_client Stream 1 type = Image resolution =
2023/07/21 16:39:15.773 7534 17828 Info ml_camera_client prepareCapture device state = Idle numStreamConfigs = 1
2023/07/21 16:39:15.773 7534 17828 Info ml_camera_client Stream 0 type = Video resolution = 1080P state = Configured
2023/07/21 16:39:15.773 7534 17828 Info ml_camera_client Stream 1 type = Image resiolution = state = Invalid
2023/07/21 16:39:15.773 7534 17828 Debug ml_camera_client PreCaptureAEAWB ++
2023/07/21 16:39:15.788 7534 7534 Warn UnityGfxDeviceW type=1400 audit(0.0:5367): avc: denied { search } for name="traces" dev="nvme0n1p37" ino=7340035 scontext=u:r:untrusted_app:s0:c105,c256,c512,c768 tcontext=u:object_r:trace_data_file:s0 tclass=dir permissive=0
2023/07/21 16:39:16.667 7534 7703 Info ml_camera_client aeawb frame listener
2023/07/21 16:39:16.667 7534 7703 Debug ml_camera_client aeawb session capturecompleted
2023/07/21 16:39:16.700 7534 7703 Info ml_camera_client aeawb frame listener
2023/07/21 16:39:16.700 7534 7552 Debug ml_camera_client aeawb session capturecompleted
2023/07/21 16:39:16.766 7534 7552 Info ml_camera_client aeawb frame listener
2023/07/21 16:39:16.767 7534 7552 Debug ml_camera_client aeawb session capturecompleted
2023/07/21 16:39:16.833 7534 7552 Info ml_camera_client aeawb frame listener
2023/07/21 16:39:16.833 7534 7552 Debug ml_camera_client aeawb session capturecompleted
2023/07/21 16:39:16.900 7534 7552 Info ml_camera_client aeawb frame listener
2023/07/21 16:39:16.900 7534 7552 Debug ml_camera_client aeawb session capturecompleted
2023/07/21 16:39:16.933 7534 7552 Info ml_camera_client aeawb frame listener
2023/07/21 16:39:16.933 7534 7703 Debug ml_camera_client aeawb session capturecompleted
2023/07/21 16:39:16.967 7534 7703 Info ml_camera_client aeawb frame listener
2023/07/21 16:39:16.967 7534 7552 Debug ml_camera_client aeawb session capturecompleted
2023/07/21 16:39:17.000 7534 7552 Info ml_camera_client aeawb frame listener
2023/07/21 16:39:17.000 7534 7552 Debug ml_camera_client aeawb session capturecompleted
2023/07/21 16:39:17.033 7534 7552 Info ml_camera_client aeawb frame listener
2023/07/21 16:39:17.033 7534 7552 Debug ml_camera_client aeawb session capturecompleted
2023/07/21 16:39:17.033 7534 7552 Debug ml_camera_client aeawb convergence signaled
2023/07/21 16:39:17.066 7534 7552 Info ml_camera_client aeawb frame listener
2023/07/21 16:39:17.067 7534 7552 Debug ml_camera_client aeawb session capturecompleted
2023/07/21 16:39:17.100 7534 7552 Info ml_camera_client aeawb frame listener
2023/07/21 16:39:17.100 7534 7552 Debug ml_camera_client aeawb session capturecompleted
2023/07/21 16:39:17.133 7534 7552 Info ml_camera_client aeawb frame listener
2023/07/21 16:39:17.133 7534 7552 Debug ml_camera_client aeawb session capturecompleted
2023/07/21 16:39:17.166 7534 7552 Info ml_camera_client aeawb frame listener
2023/07/21 16:39:17.167 7534 7552 Debug ml_camera_client aeawb session capturecompleted
2023/07/21 16:39:17.200 7534 7552 Info ml_camera_client aeawb frame listener
2023/07/21 16:39:17.200 7534 7552 Debug ml_camera_client aeawb session capturecompleted
2023/07/21 16:39:17.233 7534 7552 Info ml_camera_client aeawb frame listener
2023/07/21 16:39:17.233 7534 7552 Debug ml_camera_client aeawb session capturecompleted
2023/07/21 16:39:17.266 7534 7552 Info ml_camera_client aeawb frame listener
2023/07/21 16:39:17.266 7534 7552 Debug ml_camera_client aeawb session capturecompleted
2023/07/21 16:39:17.264 7534 7534 Warn com.concepthealth.arconsultwfvr type=1400 audit(0.0:5403): avc: denied { search } for comm=494C3243505020546872656164706F name="traces" dev="nvme0n1p37" ino=7340035 scontext=u:r:untrusted_app:s0:c105,c256,c512,c768 tcontext=u:object_r:trace_data_file:s0 tclass=dir permissive=0
2023/07/21 16:39:17.269 7534 17828 Error Camera-Device endConfigure fail Status(-8, EX_SERVICE_SPECIFIC): '3: endConfigure:739: Camera 0: Unsupported set of inputs/outputs provided'
2023/07/21 16:39:17.269 7534 17828 Info ml_camera_client startRepeatingCaptureVideo device state = Idle numStreamConfigs = 1
2023/07/21 16:39:17.269 7534 17828 Info ml_camera_client Stream 0 type = Video resolution = 1080P state = Configured
2023/07/21 16:39:17.269 7534 17828 Info ml_camera_client Stream 1 type = Image resiolution = state = Invalid
2023/07/21 16:39:17.333 7534 7703 Debug ml_camera_client got video buffer 3146 timestamp = 1012951735960
2023/07/21 16:39:17.333 7534 7552 Debug ml_camera_client capturecompletecount = 4271 timestamp = 1012951735960
2023/07/21 16:39:17.333 7534 7552 Debug ml_camera_client capture complete for 1012951735960
2023/07/21 16:39:17.335 7534 17850 Debug ml_camera_client Ready for raw video frame callback
2023/07/21 16:39:17.340 7534 17850 Debug ml_camera_client released video buffer 3147 timestamp = 1012951735960
2023/07/21 16:39:17.343 7534 7567 Info luminkarrot nova/frameworks/kali/kaliwebrtc/simple/src/Factory.cpp:106 Creating microphone audio source
2023/07/21 16:39:17.344 7534 7567 Debug luminkarrot nova/frameworks/kali/kaliwebrtc/simple/src/video/Source.cpp:186 Video track created and enabled
2023/07/21 16:39:17.344 7534 7567 Info luminkarrot nova/frameworks/kali/kaliwebrtc/simple/src/Connection.cpp:471 Add track: source = 0x7bb3d14c27d8; track id = local
2023/07/21 16:39:17.344 7534 9107 Info luminkarrot nova/frameworks/kali/kaliwebrtc/simple/src/Connection.cpp:77 Renegotiation needed in state 0
2023/07/21 16:39:17.344 7534 7567 Debug luminkarrot nova/frameworks/kali/kaliwebrtc/simple/src/audio/Source.cpp:67 Audio track created and enabled
2023/07/21 16:39:17.344 7534 7567 Info luminkarrot nova/frameworks/kali/kaliwebrtc/simple/src/Connection.cpp:471 Add track: source = 0x7bb2f27260c0; track id = audio_track
2023/07/21 16:39:17.365 7534 7567 Error CRASH *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2023/07/21 16:39:17.365 7534 7567 Error CRASH Version '2022.2.0f1 (35dcd44975df)', Build type 'Development', Scripting Backend 'il2cpp', CPU 'x86_64'
2023/07/21 16:39:17.365 7534 7567 Error CRASH Build fingerprint: 'ml/demophon_aosp/demophon:10/B3E.230526.01-R.026/29:user/release-keys'
2023/07/21 16:39:17.365 7534 7567 Error CRASH Revision: '0'
2023/07/21 16:39:17.365 7534 7567 Error CRASH ABI: 'x86_64'
2023/07/21 16:39:17.365 7534 7567 Error CRASH Timestamp: 2023-07-21 16:39:17.365339996+0000
2023/07/21 16:39:17.365 7534 7567 Error CRASH pid: 7534, tid: 7567, name: UnityMain >>> com.concepthealth.arconsultwfvr <<<
2023/07/21 16:39:17.365 7534 7567 Error CRASH uid: 10105
2023/07/21 16:39:17.365 7534 7567 Error CRASH signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr --------
2023/07/21 16:39:17.365 7534 7567 Error CRASH Cause: null pointer dereference
2023/07/21 16:39:17.365 7534 7567 Error CRASH rax 00007bb3d2aeb8a0 rbx 0000000000000000 rcx 00007bb3d2aeb8a0 rdx 00007bb360cfb030
2023/07/21 16:39:17.365 7534 7567 Error CRASH r8 0000000000000000 r9 00007bb420462300 r10 00007bb3d2aeb530 r11 0000000000000246
2023/07/21 16:39:17.365 7534 7567 Error CRASH r12 00007bb3d09b6798 r13 00007bb4186b2f60 r14 00007bb3d14c2624 r15 00007bb3d14c2618
2023/07/21 16:39:17.365 7534 7567 Error CRASH rdi 00007bb3d14c2624 rsi 00007bb3d09b6798
2023/07/21 16:39:17.365 7534 7567 Error CRASH rbp 00007bb3d14c27d8 rsp 00007bb3d2aeb820 rip 00007bb2f7beb010
2023/07/21 16:39:17.365 7534 7567 Error CRASH
2023/07/21 16:39:17.365 7534 7567 Error CRASH backtrace:
2023/07/21 16:39:17.365 7534 7567 Error CRASH #00 pc 00000000002db010 /system/lib64/liblumin_webrtc_simple.so (lumin::karrot::simple::video::Source::removeSink(lumin::karrot::simple::video::Sink*)+48) (BuildId: 3bd55de1e660953b7e7be510486d0ab6)
2023/07/21 16:39:17.365 7534 7567 Error CRASH #01 pc 00000000002da9fa /system/lib64/liblumin_webrtc_simple.so (lumin::karrot::simple::video::Sink::setSource(std::__1::shared_ptr<lumin::karrot::simple::video::Source>)+26) (BuildId: 3bd55de1e660953b7e7be510486d0ab6)
2023/07/21 16:39:17.366 7534 7552 Debug ml_camera_client got video buffer 3147 timestamp = 1012985154662
2023/07/21 16:39:17.366 7534 7551 Debug ml_camera_client capturecompletecount = 4272 timestamp = 1012985154662
2023/07/21 16:39:17.367 7534 7551 Debug ml_camera_client capture complete for 1012985154662
2023/07/21 16:39:17.368 7534 17852 Debug ml_camera_client Ready for raw video frame callback
2023/07/21 16:39:17.373 7534 17852 Debug ml_camera_client released video buffer 3148 timestamp = 1012985154662
2023/07/21 16:39:17.400 7534 7551 Debug ml_camera_client got video buffer 3148 timestamp = 1013018548538
2023/07/21 16:39:17.400 7534 7552 Debug ml_camera_client capturecompletecount = 4273 timestamp = 1013018548538
2023/07/21 16:39:17.400 7534 7552 Debug ml_camera_client capture complete for 1013018548538
2023/07/21 16:39:17.402 7534 17854 Debug ml_camera_client Ready for raw video frame callback
2023/07/21 16:39:17.404 7534 17854 Debug ml_camera_client released video buffer 3149 timestamp = 1013018548538
2023/07/21 16:39:17.433 7534 7551 Debug ml_camera_client got video buffer 3149 timestamp = 1013051669782
2023/07/21 16:39:17.466 7534 7551 Debug ml_camera_client got video buffer 3150 timestamp = 1013085051675
2023/07/21 16:39:17.499 7534 7551 Debug ml_camera_client got video buffer 3151 timestamp = 1013118403562
2023/07/21 16:39:17.510 7534 7567 Error CRASH Tombstone written to: /storage/emulated/0/Android/data/com.concepthealth.arconsultwfvr/files/tombstone_01
2023/07/21 16:39:17.533 7534 7551 Debug ml_camera_client got video buffer 3152 timestamp = 1013151904578
2023/07/21 16:39:17.566 7534 7551 Debug ml_camera_client got video buffer 3153 timestamp = 1013185335373
2023/07/21 16:39:17.600 7534 7703 Error BufferQueueConsumer [BufferItemQueue] acquireBuffer: max acquired buffer count reached: 5 (max 4)
2023/07/21 16:39:17.600 7534 7703 Error BufferItemConsumer [BufferItemQueue] Error acquiring buffer: Unknown error -38 (-38)
2023/07/21 16:39:17.600 7534 7703 Error : acquireBuffer BufferItemConsumer::acquireBuffer fail, status -38
2023/07/21 16:39:17.633 7534 7703 Error BufferQueueConsumer [BufferItemQueue] acquireBuffer: max acquired buffer count reached: 5 (max 4)
2023/07/21 16:39:17.633 7534 7703 Error BufferItemConsumer [BufferItemQueue] Error acquiring buffer: Unknown error -38 (-38)
2023/07/21 16:39:17.633 7534 7703 Error : acquireBuffer BufferItemConsumer::acquireBuffer fail, status -38
2023/07/21 16:39:17.666 7534 7703 Error BufferQueueConsumer [BufferItemQueue] acquireBuffer: max acquired buffer count reached: 5 (max 4)
2023/07/21 16:39:17.666 7534 7703 Error BufferItemConsumer [BufferItemQueue] Error acquiring buffer: Unknown error -38 (-38)
2023/07/21 16:39:17.666 7534 7703 Error : acquireBuffer BufferItemConsumer::acquireBuffer fail, status -38
2023/07/21 16:39:17.699 7534 7703 Error BufferQueueConsumer [BufferItemQueue] acquireBuffer: max acquired buffer count reached: 5 (max 4)
2023/07/21 16:39:17.700 7534 7703 Error BufferItemConsumer [BufferItemQueue] Error acquiring buffer: Unknown error -38 (-38)
2023/07/21 16:39:17.700 7534 7703 Error : acquireBuffer BufferItemConsumer::acquireBuffer fail, status -38
2023/07/21 16:39:17.733 7534 7703 Error BufferQueueConsumer [BufferItemQueue] acquireBuffer: max acquired buffer count reached: 5 (max 4)
2023/07/21 16:39:17.733 7534 7703 Error BufferItemConsumer [BufferItemQueue] Error acquiring buffer: Unknown error -38 (-38)
2023/07/21 16:39:17.733 7534 7703 Error : acquireBuffer BufferItemConsumer::acquireBuffer fail, status -38
The problem occurs when I am trying to reopen the WebRTC calling screen to restart the calling functionality within a single session of the app. This results in the application crashing. I have observed crash is coming in webrtc library.
Additionally, I've noticed that the "Camera Indicator" persists on the screen even after the WebRTC call has been disconnected and I've navigated away from the canvas that hosts the calling functionality.
Here are my questions:
- Is there a known issue with reopening the WebRTC calling screen after a disconnect? How can this situation be properly handled to avoid crashes?
- What could be causing the "Camera Indicator" to remain active after a WebRTC call has been disconnected and the canvas has been switched? How can it be correctly disabled?
Any advice or guidance on these issues would be greatly appreciated. If more details are needed regarding my application's implementation, please let me know.