Streaming the app texture from ML2 glasses to desktop user

Unity Development
,
1

Hi! We have two apps - one runs on ML2, and another on Windows. We need to stream what user #1 sees in ML2 app (NOT a video from the external cameras, but a texture from the app) to the user #2 on Windows.
We use this code to get a texture from the MagicLeap camera, specifically from our app, to stream what a user in ML2 app sees to another app user on Windows. However, when getting the texture, a critical error occurs. Below is the text of the critical error.

Unity Editor version: 2023.3.27f1
ML2 OS version: 2023.3.27
Unity SDK version: 2.5.0
Host OS: (Windows/MacOS) windows 11 x64

Error messages from logs (syntax-highlighting is supported via Markdown):

AndroidPlayer "Ml_Magic Leap 2@ADB:[192.168.1.108:0](http://192.168.1.108:0/)" ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
at System.Collections.Generic.List`1[T].get_Item (System.Int32 index) [0x00000] in <00000000000000000000000000000000>:0
at UnityEngine.Experimental.Rendering.XRPass.GetViewport (System.Int32 viewIndex) [0x00000] in <00000000000000000000000000000000>:0
at UnityEngine.Rendering.Universal.Internal.DrawObjectsPass.ExecutePass (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.Internal.DrawObjectsPass+PassData data, UnityEngine.Rendering.Universal.RenderingData& renderingData, System.Boolean yFlip) [0x00000] in <00000000000000000000000000000000>:0
at UnityEngine.Rendering.Universal.ScriptableRenderer.ExecuteRenderPass (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.ScriptableRenderPass renderPass, UnityEngine.Rendering.Universal.RenderingData& renderingData) [0x00000] in <00000000000000000000000000000000>:0
at UnityEngine.Rendering.Universal.ScriptableRenderer.ExecuteBlock (System.Int32 blockIndex, UnityEngine.Rendering.Universal.ScriptableRenderer+RenderBlocks& renderBlocks, UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.RenderingData& renderingData, System.Boolean submit) [0x00000] in <00000000000000000000000000000000>:0
at UnityEngine.Rendering.Universal.ScriptableRenderer.Execute (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.RenderingData& renderingData) [0x00000] in <00000000000000000000000000000000>:0
at UnityEngine.Rendering.Universal.UniversalRenderPipeline.RenderSingleCamera (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.CameraData& cameraData) [0x00000] in <00000000000000000000000000000000>:0
at UnityEngine.Rendering.Universal.UniversalRenderPipeline.RenderCameraStack (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Camera baseCamera) [0x00000] in <00000000000000000000000000000000>:0
at UnityEngine.Rendering.Universal.UniversalRenderPipeline.Render (UnityEngine.Rendering.ScriptableRenderContext renderContext, System.Collections.Generic.List`1[T] cameras) [0x00000] in <00000000000000000000000000000000>:0
at UnityEngine.Rendering.RenderPipelineManager.DoRenderLoop_Internal (UnityEngine.Rendering.RenderPipelineAsset pipe, System.IntPtr loopPtr, UnityEngine.Object renderRequest) [0x00000] in <00000000000000000000000000000000>:0
at Byn.Unity.Examples.VirtualCamera.Update () [0x00000] in <00000000000000000000000000000000>:0
Byn.Unity.Examples.VirtualCamera:Update()
2

Do you mind providing more information about your implementation? Based on the logs the error is coming from your implementation of streaming the viewport from the Magic Leap. Specifically Byn.Unity.Examples.VirtualCamera.Update()

Have you considered getting the image of the Virtual Content using the Magic Leap Camera API? The Main Camera Stream can be configured to stream either Mixed Reality or Virtual only content. These will display what the virtual content from the user's point of view.