Camera sometimes not released when app crashes or is closed. Can't reconnect without restarting ML2

Native Development
1

I'm having an issue with the ML2 Main/CV camera where if your app has the camera connected and streaming video, and then the app crashes (due to my code, not the magic leap's) or is stopped via Android Studio's "Stop" button, then sometimes the camera is not released by the app, and the Magic Leap 2 treats it as if it is still in-use. Once kicked back to the home screen, you can still see the green dot in the upper left of your view indicating that the camera is active, even after the app is terminated. If I relaunch my app, it fails to connect to the Main/CV camera because the resource is still being reserved/taken by the previously terminated instance of the app.

The only way I've been able to reset the camera is by restarting the magic leap 2 as a whole, but that can add quite a bit of time to debugging and development when frequently making changes and relaunching the app, or debugging a crash in my code.

Is there a way to fix this or better way to handle if the camera is detected as already in-use?

Thanks

ML2 OS version: 1.1.0
MLSDK version: 1.1.0
Host OS: Windows 10

Error messages from logs (syntax-highlighting is supported via Markdown):

2

Thanks for sharing @alex.teepe. Are you able to share any diagnostic logs that may help give us insight into what is happening? Thank you.

3

Naturally, as soon as I try to intentionally replicate the crash, it stops failing as often, but I did manage to reproduce it and capture some logs.

Unfortunately, usually when the app crashes or is terminated, there's nothing related to or relevant to the camera to indicate what might be wrong. But when relaunching the app with the camera still detected as in-use by the previous launch of the app, I get these errors in the logs

I/ml_camera_client: Camonly OnAvailable CamId = 0
I/ml_camera_client: Camonly OnAvailable CamId = 1
I/ml_camera_client: MixedRealityCamera OnAvailable
V/threaded_app: InputQueueCreated: 0x76ec6f6fe580 -- 0x76ec6f6fde00
I/com.magicleap.capi.sample.camera_preview: Main camera is available!
E/Camera-Device: connectDevice faild: Status(-8, EX_SERVICE_SPECIFIC): '1: validateClientPermissionsLocked:1075: Callers from device user 0 are not currently allowed to connect to camera "1"'
E/ml_camera_client: Failed to connect to camera device
E/com.magicleap.capi.sample.camera_preview: C:/Users/<User>/MagicLeap/mlsdk/v1.1.0/samples/c_api/samples/camera/preview/app/src/main/cpp/main.cpp:699 SetupCamera (4)MLResult_UnspecifiedFailure
E/com.magicleap.capi.sample.camera_preview: C:/Users/<User>/MagicLeap/mlsdk/v1.1.0/samples/c_api/samples/camera/preview/app/src/main/cpp/main.cpp:275 SetupRestrictedResources (4)MLResult_UnspecifiedFailure
E/com.magicleap.capi.sample.camera_preview: C:/Users/<User>/MagicLeap/mlsdk/v1.1.0/samples/c_api/samples/camera/preview/app/src/main/cpp/main.cpp:719 StartCapture (5)MLResult_InvalidParam
E/com.magicleap.capi.sample.camera_preview: C:/Users/<User>/MagicLeap/mlsdk/v1.1.0/samples/c_api/samples/camera/preview/app/src/main/cpp/main.cpp:276 SetupRestrictedResources (5)MLResult_InvalidParam
V/threaded_app: APP_CMD_INPUT_CHANGED
V/threaded_app: Attaching input queue to looper
I/com.magicleap.capi.app_framework: APP_CMD_INPUT_CHANGED
V/threaded_app: ContentRectChanged: l=0,t=75,r=720,b=920
V/threaded_app: Pause: 0x76ec6f6fe580
V/com.magicleap.capi.app_framework: Ignored App Command 5
V/threaded_app: activityState=13
I/com.magicleap.capi.app_framework: APP_CMD_PAUSE

As for some of the causes of the Camera getting stuck in-use by terminated apps, I noticed in the past that when I had the option "Compute Pack Standby" enabled in the ML2 settings, and then took off the headset while the app was running, and closed the app via Android Studio either while the headset was in standby mode or shortly after waking it back up, that was a fairly consistent way of getting the camera stuck in-use.

Another way of triggering the issue now is an issue with MLCameraUpdateCaptureSettings(), where the app will freeze the second or third time I call MLCameraUpdateCaptureSettings().

I'm trying to update the camera exposure time setting live while streaming from it.
Inside of the callback

  static void OnVideoAvailable(const MLCameraOutput *output, const MLHandle metadata_handle,
                               const MLCameraResultExtras *extra, void *data)

I'm checking if the user wants to change the camera exposure time (value set/updated somewhere else in another thread), and if so, call MLCameraMetadataSetSensorExposureTime() with the new exposure time, and then use std::async() to call MLCameraUpdateCaptureSettings(), saving the std::future into a class member variable, and also checking if a previous async call to Update the camera settings is still running and not re-calling the update function until the previous call has finished.

This works for the first 2-3 times, but future calls to update the camera capture settings causes the application to freeze, and MLCameraUpdateCaptureSettings() returns 4 (MLResult_UnspecifiedFailure)

E/Camera-Device: updateStreamingRequestList fail Status(-8, EX_SERVICE_SPECIFIC): '10: updateStreamingRequestList:427: Camera 1:  Got error Connection timed out (-110) after trying to update streaming request'
I/ml_camera_client: updateCatureSettings failed
E/com.magicleap.capi.sample.camera_preview: CameraUpdateCaptureSettings - end
E/com.magicleap.capi.sample.camera_preview: MLCameraUpdateCaptureSettings() returned 4
E/com.magicleap.capi.sample.camera_preview: C:/Users/<User>/MagicLeap/mlsdk/v1.1.0/samples/c_api/samples/camera/preview/app/src/main/cpp/main.cpp:358 OnVideoAvailable (4)MLResult_UnspecifiedFailure
I/com.magicleap.capi.sample.camera_preview: Frame info: width = 4096, height = 3072, size = 50331648, bytes per pixel = 4, pixel stride = 4
E/com.magicleap.capi.sample.camera_preview: CameraUpdateCaptureSettings - begin
I/ml_camera_client: updateCatureSettings called when device not streaming
E/com.magicleap.capi.sample.camera_preview: CameraUpdateCaptureSettings - end

Closing the app after this occurs can generally cause the camera to get stuck as well.

4

Thanks @alex.teepe. Please post the logs if you can, no need to copy and paste anything. This helps our engineers best inspect the issue.