Crash with MLCamera API (we know it has deprecated)

[noxowl]

Crash with MLCamera API (we know it has deprecated)

Give us as much detail as possible regarding the issue you're experiencing:

Unity Editor version: 2022.3.x
ML2 OS version: 1.4.0-dev2 ~ 1.4.1
Unity SDK version: 1.11.0
Host OS: (Windows/MacOS) Both

We developed the application for experiment from last year. and due to the schedule, we still using the deprecated ML API not the OpenXR API.

The crash appear in the order of long-used devices for development. The first device has been repaired due to this, and so far it seems to be working as normal. (Over an hour of use)
The devices with problem will crash at a usage time of about 10 minutes.
We using 1.4.0 until now, so when we updated the problematic device to 1.4.1 and initialized the factory, it was able to use it for more than 40 minutes, but after that, it returned to about 10 minutes of use.

This problem is also seen in sample applications using MLCamera API.

Error messages from logs (syntax-highlighting is supported via Markdown):

https://gist.github.com/noxowl/b75f808e6275be8687779abe131f11b7

[kbabilinski]

I'm not seeing anything directly in the logs, I also have not been able to reproduce the crash inside the Magic Leap Camera Capture example in Unity. Do you mind testing with SDK version 1.12.0? Are there any steps that I am missing when testing inside the Unity Example Project?

[noxowl]

Thank you for your help.

The attached log is very long, so you will need to view it in RAW. There is an error log related to Buffer.

We will share our tests using Unity SDK 1.12 after Monday Japan time.

Also, this issue is only seen on certain devices, so we can only reproduce it on the specific device where we encountered the issue.

However, if this device does not use the MLCamera API, it able to use for over than 10 min.

[kbabilinski]

Additionally, do you mind testing on the newer version of the Magic Leap OS?

[noxowl]

Good afternoon.

We attached 4 log data that contains:
https://gist.github.com/noxowl/3d0a1b25fd1d15021af25b6d2113e7ff

1. ML OS 1.9 with our app (Unity SDK 1.12.0). that contains crash.
2. ML OS 1.4.1 with our app (Unity SDK 1.12.0). that fresh factory reset and no crash.
3. ML OS 1.4.1 with our app (Unity SDK 1.12.0). that contains crash.
4. ML OS 1.4.1 with sample app (Unity SDK 1.12.0). that contains crash.

We hope it will help us solve our problem.

[kbabilinski]

Thank you for that information. Do you mind providing more details about your application? You said that you were able to reproduce this in the Magic Leap Examples CVCamera scene?

[kbabilinski]

I have not been able to reproduce it in that scene and ran the example for 20 minutes.

[noxowl]

Our application is built with the MLCamera API and the Unity WebRTC plugin.
here is a code snippet that represent how we use this API.

Observable.FromCoroutine(WaitUntilMLCameraAvailable)
                    .Subscribe(_ => { }, () =>
                    {
                        MLCamera.ConnectContext connectContext = MLCamera.ConnectContext.Create();
                        connectContext.CamId = MLCamera.Identifier.Main;
                        connectContext.Flags = MLCamera.ConnectFlag.CamOnly;
                        connectContext.EnableVideoStabilization = true;
                        _mlCamera = MLCamera.CreateAndConnect(connectContext);
                        if (_mlCamera != null)
                        {
                            Debug.LogFormat("[WebRTC Steamer Log] MLCamera connected");
                        }
                    }).AddTo(this);
                if (_mlCamera != null)
                {
                    MLResult result = _mlCamera.PrepareCapture(
                        new MLCamera.CaptureConfig
                        {
                            CaptureFrameRate = MLCameraBase.CaptureFrameRate._30FPS,
                            StreamConfigs = new MLCameraBase.CaptureStreamConfig[]
                            {
                                MLCameraBase.CaptureStreamConfig.Create(
                                    new MLCameraBase.StreamCapability
                                    {
                                        CaptureType = MLCameraBase.CaptureType.Video,
                                        Width = width,
                                        Height = height,
                                    },
                                    MLCameraBase.OutputFormat.RGBA_8888,
                                    null)
                            }
                        }, out MLCamera.Metadata metadata);
...

Furthermore, the sample application that reproduces the problem is the WorldCamera scene.
We haven't checked CVCamera, but I think it will be the same.

We currently have three ML2 devices, and I believe it's an important point that the issue is only reproducible on the one that has been used the longest.

[kbabilinski]

Interesting, I will test with the world camera demo.

Do you know if that device was upgraded following the upgrade path specified on the customer care website. I wonder if you are running into the same issue as this developer.

Issue with Marker Tracking App on Magic Leap 2

We are tracking a similar issue with another developer. We currently think it is related to jumping too many version of the OS at once. To note, this process above is not a workaround and is documented here. However, we are doing further testing to see if when a device is updated using the prescribed process, the reboot behavior is reduced/resolved Rollback devices to 1.1.0 (check the "factory reset option, only during this step. Please leave unchecked for steps 2-4) Update 1.1.0 to 1.2.0 Upd…

[noxowl]

Thank you for the solution.

But We have to tell you the bad news.
The same issue occurs despite the above update process.

In this analysis, we thought that audio processing might be related to crash.

We have attached two log data:

1. ML OS 1.9 with our app (Unity SDK 2.4.0 use only for compilation. the code base is for 1.11).
2. ML OS 1.9 with sample app (Unity SDK 2.4.0). and switching scenes such as WorldCamera, CVCamera, AudioCapture.

https://gist.github.com/noxowl/94220196a3ddbc740688d1e4c87d20b0

[kbabilinski]

Thank you for sharing this information with us. Question regarding reproducing this issue in the Camera Example. Does the issue occur when using the MLSDK example's CVCamera scene and then switching to the WorldCamera example scene?

[noxowl]

It occur randomly. such as switching between WorldCamera, CVCamera, AudioCapture (sorry for the inaccurate name—I'm currently working on a new application using the latest ML2 SDK), and back to CVCamera, and so on.

However, the key difference is that our application causes the device to reboot (it reboots after the 'tracking lost' message), while the sample app only causes the application to close.

It would be great if this could definitively narrow down the issue. Unfortunately, in the past, there were instances where only the application would close without causing the device to reboot.

And our issue is in the same line as the one previously reported(but the code base is not same. the common aspect is that both use the MLCamera API.). Could this be a clue to solving the problem?

Freezing in a camera-recording application

Dear @etucker and @noxowl , The MLCamera problem happens again in my another magic leap. Is there any way to solve this?

Additionally, we faced new issue with MLCamera API that reported before. Would it be better to report this in a new thread?

Permission failure: com.magicleap.permission.CAMERA_CAPTURE_CVIP

Hi, I'm getting this error "Permission failure: com.magicleap.permission.CAMERA_CAPTURE_CVIP" in my log despite having camera permissions in my AndroidManifest. I cannot find this permission anywhere, not even in documentation. I even added that specific permission to my xml. What am I doing wrong? All versions of sdks are latest.

Here is the log sample:
Environment: ML OS 1.9 + Unity SDK 2.4.0

10-18 14:57:11.769 24573 24594 I Unity   : [WebRTC Steamer Log] CaptureVideoStart
10-18 14:57:11.771 24573 24594 I CameraManagerGlobal: Connecting to camera service
10-18 14:57:11.772  3319  3701 W ServiceManager: Permission failure: com.magicleap.permission.CAMERA_CAPTURE_CVIP from uid=10108 pid=24573
10-18 14:57:11.772  3319  3701 D ServiceManager: Permission check violation (permissive=0): com.magicleap.permission.CAMERA_CAPTURE_CVIP from uid=10108 pid=24573, Permission ARE enforcing.
10-18 14:57:11.773  3319  3701 W ServiceManager: Permission failure: com.magicleap.permission.CAMERA_CAPTURE_CVIP from uid=10108 pid=24573
10-18 14:57:11.773  3319  3701 D ServiceManager: Permission check violation (permissive=0): com.magicleap.permission.CAMERA_CAPTURE_CVIP from uid=10108 pid=24573, Permission ARE enforcing.
10-18 14:57:11.774  3319  3701 W ServiceManager: Permission failure: com.magicleap.permission.CAMERA_CAPTURE_CVIP from uid=10108 pid=24573
...
10-18 14:57:11.781 24573 24594 I Unity   : [WebRTC Steamer Log] WebCam device found
10-18 14:57:11.783 24573 24594 I Unity   : [WebRTC Steamer Log] Available webcam: Camera 0
10-18 14:57:11.783 24573 24594 I Unity   : O2M.AppleCider.Network.WebRTC.<>c:<CaptureVideoStart>b__30_0(WebCamDevice)
10-18 14:57:11.783 24573 24594 I Unity   : System.Collections.Generic.List`1:ForEach(Action`1)
10-18 14:57:11.783 24573 24594 I Unity   : O2M.AppleCider.Network.WebRTC.<CaptureVideoStart>d__30:MoveNext()
10-18 14:57:11.783 24573 24594 I Unity   : UniRx.<WrapEnumerator>d__291:MoveNext()
10-18 14:57:11.783 24573 24594 I Unity   : UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
10-18 14:57:11.783 24573 24594 I Unity   : UniRx.InternalUtil.ThreadSafeQueueWorker:ExecuteAll(Action`1)
10-18 14:57:11.783 24573 24594 I Unity   : 
10-18 14:57:11.784 24573 24594 I Unity   : [WebRTC Steamer Log] Available webcam: Camera 1
10-18 14:57:11.784 24573 24594 I Unity   : O2M.AppleCider.Network.WebRTC.<>c:<CaptureVideoStart>b__30_0(WebCamDevice)
10-18 14:57:11.784 24573 24594 I Unity   : System.Collections.Generic.List`1:ForEach(Action`1)
10-18 14:57:11.784 24573 24594 I Unity   : O2M.AppleCider.Network.WebRTC.<CaptureVideoStart>d__30:MoveNext()
10-18 14:57:11.784 24573 24594 I Unity   : UniRx.<WrapEnumerator>d__291:MoveNext()
10-18 14:57:11.784 24573 24594 I Unity   : UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
10-18 14:57:11.784 24573 24594 I Unity   : UniRx.InternalUtil.ThreadSafeQueueWorker:ExecuteAll(Action`1)
10-18 14:57:11.784 24573 24594 I Unity   : 
10-18 14:57:11.784 24573 24594 I Unity   : [WebRTC Steamer Log] Available webcam: Camera 2
10-18 14:57:11.784 24573 24594 I Unity   : O2M.AppleCider.Network.WebRTC.<>c:<CaptureVideoStart>b__30_0(WebCamDevice)
10-18 14:57:11.784 24573 24594 I Unity   : System.Collections.Generic.List`1:ForEach(Action`1)
10-18 14:57:11.784 24573 24594 I Unity   : O2M.AppleCider.Network.WebRTC.<CaptureVideoStart>d__30:MoveNext()
10-18 14:57:11.784 24573 24594 I Unity   : UniRx.<WrapEnumerator>d__291:MoveNext()
10-18 14:57:11.784 24573 24594 I Unity   : UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
10-18 14:57:11.784 24573 24594 I Unity   : UniRx.InternalUtil.ThreadSafeQueueWorker:ExecuteAll(Action`1)
10-18 14:57:11.784 24573 24594 I Unity   : 
10-18 14:57:11.821 24573 24653 W Android_FMOD_MLAudio: OutputMLAudio::recordLock, offset 5696, length 320, mRecordPositionBytes 0
10-18 14:57:11.821 24573 24653 W Android_FMOD_MLAudio: OutputMLAudio::recordUnlock, mRecordPositionBytes 0
10-18 14:57:11.821 24573 24653 W Android_FMOD_MLAudio: OutputMLAudio::recordLock, offset 6016, length 320, mRecordPositionBytes 0
10-18 14:57:11.821 24573 24653 W Android_FMOD_MLAudio: OutputMLAudio::recordUnlock, mRecordPositionBytes 0

[kbabilinski]

Thank you for that information.

The random reboot can suggest that there is a memory leak somewhere. Would it be possible for you to upgrade your application to OpenXR? You will still be able to use the MLCamera API as you did previously.

---

I will test the steps that you mentioned regarding the World Camera, CVCamera and Audio Capture.

---

Regarding the the com.magicleap.permission.CAMERA_CAPTURE_CVIP logs, these are just a warning related to a system function that are un related to your application.

[kbabilinski]

I just tested the Unity 1.12.1 Unity Example project and unfortunately was unable to get the crash that you mentioned.

Can you verify that your application is disposing it's resources correctly? You can also try to profile your app to see if there are any undisposed references.