ML2 Audio Recording Crash in Unity 2022-2.0f1 - Seeking Assistance

[usman.bashir]

Hello fellow developers,

I'm currently experiencing an issue with audio recording in our Unity-based ML2 application. I'm reaching out to the community in hopes of finding a resolution or getting guidance on how to proceed.

Environment Details:

• Unity Editor version: 2022-2.0f1
• ML2 OS version: 1.3.0-dev2
• MLSDK version: 1.7.0
• Host OS: Windows 11

Issue:

While attempting to record audio, the application crashes. The stack trace seems to point towards the native audio recording APIs, specifically involving calls into libaudio.

Error Logs:

0001/01/01 00:00:00.000 -1 -1 Info  --------- beginning of crash
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime FATAL EXCEPTION: UnityMain
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime Process: com.concepthealth.arconsult, PID: 12116
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime java.lang.Error: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime Version '2022.2.0f1 (35dcd44975df)', Build type 'Development', Scripting Backend 'il2cpp', CPU 'x86_64'
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime Build fingerprint: 'ml/demophon_aosp/demophon:10/B3E.230526.01-R.026/29:user/release-keys'
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime Revision: '0'
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime ABI: 'x86_64'
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime Timestamp: 2023-10-26 18:16:49.117038419+0000
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime pid: 12116, tid: 12139, name: Binder:12116_4  >>> com.concepthealth.arconsult <<<
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime uid: 10127
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime     rax 0000000000000000  rbx 0000000000002f54  rcx 0000715b018233f8  rdx 0000000000000006
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime     r8  0000715a23a75260  r9  0000000000000000  r10 0000715a23a75220  r11 0000000000000246
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime     r12 0000715a23a75470  r13 0000003000000008  r14 0000715a23a752b8  r15 0000000000002f6b
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime     rdi 0000000000002f54  rsi 0000000000002f6b
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime     rbp 0000715a23a75330  rsp 0000715a23a75218  rip 0000715b018233f8
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime backtrace:
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime       #00 pc 00000000000983f8  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+24) (BuildId: d7be87af08c63b7517a994aa40288e45)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime       #01 pc 000000000009b042  /apex/com.android.runtime/lib64/bionic/libc.so (abort+178) (BuildId: d7be87af08c63b7517a994aa40288e45)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libunity.backtrace(void**, int)(backtrace:36)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libunity.NativeRuntimeException::SignalHandler(int, siginfo*, void*)(SignalHandler:210)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libc.__restore_rt(__restore_rt:0)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libc.syscall(syscall:24)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libc.abort(abort:179)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libil2cpp.0x1ad82bc(Native Method)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libil2cpp.0x1ad84c7(Native Method)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libil2cpp.0x1ad8363(Native Method)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libil2cpp.0x1ad7a66(Native Method)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libil2cpp.0x1ad79ed(Native Method)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libil2cpp.0x1a4319e(Native Method)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libil2cpp.0x1a48b3e(Native Method)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libil2cpp.0x29edad6(Native Method)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libil2cpp.0x29ee4cc(Native Method)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libil2cpp.0x29edfe3(Native Method)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libil2cpp.0x196010c(Native Method)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libaudio.0x240dc(Native Method)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libaudio.0x49c91(Native Method)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libbinder.android::BBinder::transact(unsigned int, android::Parcel const&, android::Parcel*, unsigned int)(transact:95)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libbinder.android::IPCThreadState::executeCommand(int)(executeCommand:1112)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libbinder.android::IPCThreadState::getAndExecuteCommand()(getAndExecuteCommand:158)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libbinder.android::IPCThreadState::joinThreadPool(bool)(joinThreadPool:46)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libbinder.0x8d148(Native Method)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libutils.android::Thread::_threadLoop(void*)(_threadLoop:346)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libandroid_runtime.android::AndroidRuntime::javaThreadShell(void*)(javaThreadShell:132)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libc.__pthread_start(void*)(__pthread_start:31)
2023/10/26 18:16:49.372 12116 12138 Error AndroidRuntime 	at libc.__start_thread(__start_thread:56)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime ro.java.core is set, making core file..
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime java.lang.Error: FATAL EXCEPTION [UnityMain]
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime Unity version     : 2022.2.0f1
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime Device model      : Magic Leap Magic Leap 2
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime Device fingerprint: ml/demophon_aosp/demophon:10/B3E.230526.01-R.026/29:user/release-keys
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime CPU supported ABI : [x86_64, x86]
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime Build Type        : Development
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime Scripting Backend : IL2CPP
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime Libs loaded from  : lib/x86_64
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime Strip Engine Code : true
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime Caused by: java.lang.Error: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime Version '2022.2.0f1 (35dcd44975df)', Build type 'Development', Scripting Backend 'il2cpp', CPU 'x86_64'
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime Build fingerprint: 'ml/demophon_aosp/demophon:10/B3E.230526.01-R.026/29:user/release-keys'
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime Revision: '0'
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime ABI: 'x86_64'
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime Timestamp: 2023-10-26 18:16:49.117038419+0000
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime pid: 12116, tid: 12139, name: Binder:12116_4  >>> com.concepthealth.arconsult <<<
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime uid: 10127
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime     rax 0000000000000000  rbx 0000000000002f54  rcx 0000715b018233f8  rdx 0000000000000006
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime     r8  0000715a23a75260  r9  0000000000000000  r10 0000715a23a75220  r11 0000000000000246
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime     r12 0000715a23a75470  r13 0000003000000008  r14 0000715a23a752b8  r15 0000000000002f6b
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime     rdi 0000000000002f54  rsi 0000000000002f6b
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime     rbp 0000715a23a75330  rsp 0000715a23a75218  rip 0000715b018233f8
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime backtrace:
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime       #00 pc 00000000000983f8  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+24) (BuildId: d7be87af08c63b7517a994aa40288e45)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime       #01 pc 000000000009b042  /apex/com.android.runtime/lib64/bionic/libc.so (abort+178) (BuildId: d7be87af08c63b7517a994aa40288e45)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libunity.backtrace(void**, int)(backtrace:36)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libunity.NativeRuntimeException::SignalHandler(int, siginfo*, void*)(SignalHandler:210)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libc.__restore_rt(__restore_rt:0)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libc.syscall(syscall:24)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libc.abort(abort:179)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libil2cpp.0x1ad82bc(Native Method)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libil2cpp.0x1ad84c7(Native Method)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libil2cpp.0x1ad8363(Native Method)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libil2cpp.0x1ad7a66(Native Method)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libil2cpp.0x1ad79ed(Native Method)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libil2cpp.0x1a4319e(Native Method)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libil2cpp.0x1a48b3e(Native Method)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libil2cpp.0x29edad6(Native Method)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libil2cpp.0x29ee4cc(Native Method)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libil2cpp.0x29edfe3(Native Method)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libil2cpp.0x196010c(Native Method)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libaudio.0x240dc(Native Method)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libaudio.0x49c91(Native Method)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libbinder.android::BBinder::transact(unsigned int, android::Parcel const&, android::Parcel*, unsigned int)(transact:95)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libbinder.android::IPCThreadState::executeCommand(int)(executeCommand:1112)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libbinder.android::IPCThreadState::getAndExecuteCommand()(getAndExecuteCommand:158)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libbinder.android::IPCThreadState::joinThreadPool(bool)(joinThreadPool:46)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libbinder.0x8d148(Native Method)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libutils.android::Thread::_threadLoop(void*)(_threadLoop:346)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libandroid_runtime.android::AndroidRuntime::javaThreadShell(void*)(javaThreadShell:132)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libc.__pthread_start(void*)(__pthread_start:31)
2023/10/26 18:16:49.373 12116 12138 Error AndroidRuntime 	at libc.__start_thread(__start_thread:56)
0001/01/01 00:00:00.000 -1 -1 Info  --------- beginning of main
2023/10/26 18:17:29.031 12116 12116 Verbose MediaRouter Selecting route: RouteInfo{ name=Phone, description=null, status=null, category=RouteCategory{ name=System types=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO  groupable=false }, supportedTypes=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO , presentationDisplay=null }

Additional Information:

• Microphone permissions are verified and enabled.
• I've thoroughly reviewed the script and configuration, but I'm unable to identify the source of the issue.

I am posting detailed logs of the app, bug report to this email and my current script for your assistance.

Recording.zip (88.9 KB)

Can you please quickly respond to this email as we are in a bit hurry?

Kind regards,

Muhammad Usman

[kbabilinski]

Hi, thank you for posting your question on the developer forum. To help you better, we need some more information about your issue. Please answer the following questions:

• Did you modify the Audio Capture example in any way? If yes, what changes did you make and why?
• What are you trying to achieve with your project? Is there a specific reason why you are not using the standard Unity Microphone API?
• Can you reproduce the bug in the Unity example project and share the steps with us? This will help us to identify the cause of the crash and provide a solution.

Without the reproduction steps, it is very difficult to debug your issue. It is like finding a needle in a haystack. Please provide as much detail as possible so we can assist you better. Thank you for your cooperation.

[usman.bashir]

I am recording Audio to a wav file. On Start recording, I basically start capturing the audio, and on stop recording, I am saving audio buffer to local file.

[kbabilinski]

Do you mind providing the additional information requested above?

• Did you modify the Audio Capture example script? Can you explain/ show the changes?
• Have you considered using the standard Unity Microphone API?
• Can you reproduce the bug in the Unity example project and share the steps with us?

This will help us to identify the cause of the crash and provide a solution.

[usman.bashir]

@kbabilinski Just to inform you, I have already attached zip file for your reference in my question. Please find some time to visit it.

Answer to First Question:

I took motivation from AudioCaptureExample as well as the documentation reference which you sent over email.

• Permissions: The app first requests audio recording permissions.
  Start Recording:
  A filename is generated based on the current time.
• An audio buffer (MLAudioInput.BufferClip) is initialized to capture voice audio.
  Stop Recording:
• The recorded audio data in the buffer is saved to a WAV file using the SaveToFile and SaveToWav methods.

Saving Audio:

• The SaveToWav method utilizes the WavUtility to convert the captured audio (AudioClip) into WAV format and save it to the desired path.

Key Audio Objects:

• MLAudioInput.BufferClip: Captures audio.
• AudioClip: Holds the recorded audio data.

Answer to Second Question:
I tried initially but it didn't worked for me. For ML2 development, I mostly focus on Magic leap sdks.

Answer to third Question:

It will take more time for me to reproduce the same using unity APIs, as I am already finalizing other features of the app.

[kbabilinski]

Im not familiar with the WavUtility function. Is this a function you created or part of a plugin. I suggest using break points or logs to determine at which point the script breaks. I also recommend creating a simple script to test your implementation of the audio clip to Wav conversion. This will help narrow down what is happening.

[usman.bashir]

Script breaks in "StartRecording" function, and it is clearly mentioned in the log file which I have posted in the ZIP file:

Here are initial crash logs:

2023/10/26 17:55:56.576 6305 6329 Info Unity Geocoding latitude: 31.68875; Longitude: -106.2403
2023/10/26 17:55:56.576 6305 6329 Info Unity <>c__DisplayClass12_0:<UpdateUI>b__0(Single, Single)
2023/10/26 17:55:56.576 6305 6329 Info Unity <Geocoding>d__13:MoveNext()
2023/10/26 17:55:56.576 6305 6329 Info Unity UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
2023/10/26 17:55:56.576 6305 6329 Info Unity UnityEngine.Events.UnityEvent`1:Invoke(T0)
2023/10/26 17:55:56.576 6305 6329 Info Unity GameEvent_Appointment:Raise(Appointment)
2023/10/26 17:55:56.576 6305 6329 Info Unity UnityEngine.Events.UnityEvent:Invoke()
2023/10/26 17:55:56.576 6305 6329 Info Unity UnityEngine.EventSystems.ExecuteEvents:Execute(GameObject, BaseEventData, EventFunction`1)
2023/10/26 17:55:56.576 6305 6329 Info Unity <ClickAfterDuration>d__7:MoveNext()
2023/10/26 17:55:56.576 6305 6329 Info Unity UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
2023/10/26 17:55:56.576 6305 6329 Info Unity 
2023/10/26 17:55:59.165 6305 6329 Error Unity [RecordAudioUI] Inside StartRecording Function
2023/10/26 17:55:59.165 6305 6329 Error Unity MagicLeap.Examples.RecordAudioUI:StartRecording()
2023/10/26 17:55:59.165 6305 6329 Error Unity UnityEngine.Events.UnityEvent:Invoke()
2023/10/26 17:55:59.165 6305 6329 Error Unity GameEvent:Raise()
2023/10/26 17:55:59.165 6305 6329 Error Unity UnityEngine.Events.UnityEvent:Invoke()
2023/10/26 17:55:59.165 6305 6329 Error Unity UnityEngine.EventSystems.ExecuteEvents:Execute(GameObject, BaseEventData, EventFunction`1)
2023/10/26 17:55:59.165 6305 6329 Error Unity <ClickAfterDuration>d__7:MoveNext()
2023/10/26 17:55:59.165 6305 6329 Error Unity UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
2023/10/26 17:55:59.165 6305 6329 Error Unity 
2023/10/26 17:55:59.212 6305 6325 Error libc++abi terminating with uncaught exception of type Il2CppExceptionWrapper
2023/10/26 17:55:59.227 6305 6325 Error CRASH *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2023/10/26 17:55:59.227 6305 6325 Error CRASH Version '2022.2.0f1 (35dcd44975df)', Build type 'Development', Scripting Backend 'il2cpp', CPU 'x86_64'
2023/10/26 17:55:59.227 6305 6325 Error CRASH Build fingerprint: 'ml/demophon_aosp/demophon:10/B3E.230526.01-R.026/29:user/release-keys'
2023/10/26 17:55:59.227 6305 6325 Error CRASH Revision: '0'
2023/10/26 17:55:59.227 6305 6325 Error CRASH ABI: 'x86_64'
2023/10/26 17:55:59.227 6305 6325 Error CRASH Timestamp: 2023-10-26 17:55:59.227580235+0000
2023/10/26 17:55:59.227 6305 6325 Error CRASH pid: 6305, tid: 6325, name: Binder:6305_3  >>> com.concepthealth.arconsult <<<
2023/10/26 17:55:59.227 6305 6325 Error CRASH uid: 10126
2023/10/26 17:55:59.227 6305 6325 Error CRASH signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
2023/10/26 17:55:59.227 6305 6325 Error CRASH     rax 0000000000000000  rbx 00000000000018a1  rcx 0000715b018233f8  rdx 0000000000000006
2023/10/26 17:55:59.227 6305 6325 Error CRASH     r8  0000715a2b1a5260  r9  0000000000000000  r10 0000715a2b1a5220  r11 0000000000000246
2023/10/26 17:55:59.227 6305 6325 Error CRASH     r12 0000715a2b1a5470  r13 0000003000000008  r14 0000715a2b1a52b8  r15 00000000000018b5
2023/10/26 17:55:59.227 6305 6325 Error CRASH     rdi 00000000000018a1  rsi 00000000000018b5
2023/10/26 17:55:59.227 6305 6325 Error CRASH     rbp 0000715a2b1a5330  rsp 0000715a2b1a5218  rip 0000715b018233f8
2023/10/26 17:55:59.227 6305 6325 Error CRASH 
2023/10/26 17:55:59.227 6305 6325 Error CRASH backtrace:
2023/10/26 17:55:59.227 6305 6325 Error CRASH       #00 pc 00000000000983f8  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+24) (BuildId: d7be87af08c63b7517a994aa40288e45)
2023/10/26 17:55:59.227 6305 6325 Error CRASH       #01 pc 000000000009b042  /apex/com.android.runtime/lib64/bionic/libc.so (abort+178) (BuildId: d7be87af08c63b7517a994aa40288e45)

@kbabilinski

Recording Audio: The log mentions [RecordAudioUI] Inside StartRecording Function which suggests that the app might have a feature or a section where it tries to record audio.

Error and Crash: The application encounters an error when it tries to execute the StartRecording function and then subsequently crashes. This suggests that there might be an issue with how the application is trying to access the device's audio recording capabilities, or perhaps an underlying issue in the MagicLeap or Unity API it's relying on.

[usman.bashir]

@kbabilinski Can you please download this ZIP file?

I have attached:

1. Magic Leap Hub Bug Report
2. Unity Editor Console Logs of the App
3. RecordUI.cs script which I am using for recording

Recording.zip (88.9 KB)

[kbabilinski]

I downloaded the zip file but I’m unable to reproduce the issue on my end. I get a bunch of missing references when I import it into the Magic Leap example project. It would be helpful if you could provide a sample script that demonstrates the MLAudio API failing without the additional libraries so we can investigate further. I would also like to encourage you to consider using the standard Unity Microphone API as we are slowly transitioning to more common Android API for both video and audio.

[usman.bashir]

I have refactored the script for you:

using System;
using System.IO;
using UnityEngine;
using UnityEngine.XR.MagicLeap;

namespace MagicLeap.Examples
{
    public class RecordAudioUI : MonoBehaviour
    {
        private const string TAG = "[RecordAudioUI]";
        private const int AUDIO_CLIP_LENGTH_SECONDS = 3600;
        private const int AUDIO_CLIP_FREQUENCY_HERTZ = 48000;

        private bool isCapturing = false;
        private string deviceMicrophone = string.Empty;
        private MLAudioInput.BufferClip mlAudioBufferClip;


        private bool isRecording = false;
        private string wavFilePath;

        private readonly MLPermissions.Callbacks permissionCallbacks = new MLPermissions.Callbacks();


        private static string fileName;
        private static string startTime;
        private static int startTimeInMillis;
        private static string endTime;
        private static int endTimeInMillis;
        private static string audioDuration;

        private const string validFileFormat = ".wav";


        void Awake()
        {
            permissionCallbacks.OnPermissionGranted += OnPermissionGranted;
            permissionCallbacks.OnPermissionDenied += OnPermissionDenied;
            permissionCallbacks.OnPermissionDeniedAndDontAskAgain += OnPermissionDenied;

            MLPermissions.RequestPermission(MLPermission.RecordAudio, permissionCallbacks);

            dataController = GameObject.FindObjectOfType<DataController>();
        }

        void OnDestroy()
        {
            StopRecording();
            permissionCallbacks.OnPermissionGranted -= OnPermissionGranted;
            permissionCallbacks.OnPermissionDenied -= OnPermissionDenied;
            permissionCallbacks.OnPermissionDeniedAndDontAskAgain -= OnPermissionDenied;
        }

        //public void StartRecording()
        //{
        //    DateTime currentTime = DateTime.Now;

        //    fileName = "audio_" + currentTime.ToString("yyyyMMdd_HHmmss");
        //    startTime = currentTime.ToString("HH:mm:ss");
        //    startTimeInMillis = currentTime.Millisecond;
        //    //wavFilePath = System.IO.Path.Combine(Application.persistentDataPath, fileName);
        //    wavFilePath = System.IO.Path.Combine("/storage/emulated/0/Android/data/com.concepthealth.arconsult/files", fileName);

        //    Debug.LogError(TAG + " Inside StartRecording Function");

        //    if (!MLPermissions.CheckPermission(MLPermission.RecordAudio).IsOk)
        //    {
        //        Debug.LogError(TAG + $"StartRecording() cannot start, {MLPermission.RecordAudio} not granted.");
        //        return;
        //    }

        //    if (isCapturing)
        //    {
        //        Debug.LogWarning(TAG + "Already capturing audio!");
        //        return;
        //    }

        //    isCapturing = true;
        //    mlAudioBufferClip = new MLAudioInput.BufferClip(MLAudioInput.MicCaptureType.VoiceCapture, AUDIO_CLIP_LENGTH_SECONDS, MLAudioInput.GetSampleRate(MLAudioInput.MicCaptureType.VoiceCapture));

        //}


        public void StartRecording()
        {
            try
            {
                Debug.Log(TAG + " Attempting to start recording...");

                if (isCapturing)
                {
                    Debug.LogWarning(TAG + " Recording is already in progress.");
                    return;
                }

                if (!MLPermissions.CheckPermission(MLPermission.RecordAudio).IsOk)
                {
                    Debug.LogError(TAG + $"Cannot start recording as {MLPermission.RecordAudio} has not been granted.");
                    return;
                }

                DateTime currentTime = DateTime.Now;

                if (currentTime == null)
                {
                    Debug.LogError(TAG + " Failed to get the current time.");
                    return;
                }

                fileName = "audio_" + currentTime.ToString("yyyyMMdd_HHmmss");
                startTime = currentTime.ToString("HH:mm:ss");
                startTimeInMillis = currentTime.Millisecond;

                if (string.IsNullOrEmpty(fileName))
                {
                    Debug.LogError(TAG + " Failed to create a filename.");
                    return;
                }

                wavFilePath = System.IO.Path.Combine(Application.persistentDataPath, fileName);

                if (string.IsNullOrEmpty(wavFilePath))
                {
                    Debug.LogError(TAG + " Failed to set the audio file path.");
                    return;
                }

                Debug.Log(TAG + " Preparing audio buffer...");
                mlAudioBufferClip = new MLAudioInput.BufferClip(MLAudioInput.MicCaptureType.VoiceCapture, AUDIO_CLIP_LENGTH_SECONDS, MLAudioInput.GetSampleRate(MLAudioInput.MicCaptureType.VoiceCapture));

                if (mlAudioBufferClip == null)
                {
                    Debug.LogError(TAG + " Failed to initialize audio buffer.");
                    return;
                }

                isCapturing = true;
                Debug.Log(TAG + " Recording has started successfully.");
            }
            catch (Exception ex)
            {
                Debug.LogError(TAG + " An error occurred while attempting to start recording: " + ex.Message);
            }
        }

        public void StopRecording()
        {
            Debug.Log(TAG + " Inside StopRecording Function");
            if (!isCapturing)
            {
                return;
            }

            mlAudioBufferClip?.Dispose();

            // Save the audio data to a file before disposing the buffer and uploading.
            SaveToFile(wavFilePath, AudioFileType.WAV); // Added this line to save the audio data.


            mlAudioBufferClip = null;

            isCapturing = false;


            DateTime currentTime = DateTime.Now;

            endTime = currentTime.ToString("HH:mm:ss");
            endTimeInMillis = currentTime.Millisecond;

            audioDuration = (endTimeInMillis - startTimeInMillis).ToString();

        }

        public void SaveToFile(string path, AudioFileType type)
        {
            if (mlAudioBufferClip == null)
            {
                Debug.LogError(TAG + " No audio data available to save!");
                return;
            }

            var audioClip = mlAudioBufferClip.FlushToClip();

            switch (type)
            {
                case AudioFileType.WAV:
                    SaveToWav(path, audioClip);
                    break;
                default:
                    Debug.LogError(TAG + " Invalid or unsupported file type specified!");
                    break;
            }
        }

        private void SaveToWav(string filename, AudioClip clip)
        {
            var filepath = Path.Combine(Application.persistentDataPath, filename + ".wav");

            byte[] audioBytes = WavUtility.FromAudioClip(clip);
            File.WriteAllBytes(filepath, audioBytes);

            Debug.Log(TAG + $" Saved to {filepath}");
        }

        private void OnPermissionDenied(string permission)
        {
            Debug.LogError(TAG + $" Failed to get requested permission {permission}, disabling script.");
            enabled = false;
            return;
        }

        private void OnPermissionGranted(string permission)
        {
            Debug.Log(TAG + $" Succeeded in requesting {permission}.");
        }
    }

    public enum AudioFileType
    {
        WAV
    }

    // This is a basic WAV utility for AudioClip
    public static class WavUtility
    {
        const uint HEADER_SIZE = 44;

        public static byte[] FromAudioClip(AudioClip clip)
        {
            MemoryStream stream = new MemoryStream();
            using (BinaryWriter writer = new BinaryWriter(stream))
            {
                uint totalSampleCount = (uint)(clip.samples * clip.channels);
                uint byteCount = totalSampleCount * sizeof(short);
                uint fileSize = HEADER_SIZE + byteCount;

                writer.Write("RIFF".ToCharArray());
                writer.Write(fileSize);
                writer.Write("WAVE".ToCharArray());
                writer.Write("fmt ".ToCharArray());
                writer.Write(16);
                writer.Write((ushort)1);
                writer.Write((ushort)clip.channels);
                writer.Write(clip.frequency);
                writer.Write(clip.frequency * 2);
                writer.Write((ushort)2);
                writer.Write((ushort)16);
                writer.Write("data".ToCharArray());
                writer.Write(byteCount);

                float[] samples = new float[clip.samples];
                clip.GetData(samples, 0);

                Int16[] intData = new Int16[samples.Length];
                byte[] bytesData = new byte[samples.Length * 2];

                int rescaleFactor = 32767; //to convert float to Int16

                for (int i = 0; i < samples.Length; i++)
                {
                    intData[i] = (short)(samples[i] * rescaleFactor);
                    byte[] byteArr = BitConverter.GetBytes(intData[i]);
                    byteArr.CopyTo(bytesData, i * 2);
                }

                writer.Write(bytesData);
            }

            return stream.ToArray();
        }
    }
}

Now, you have to call StartRecording and StopRecording functions by your own.

Secondly, one of my colleagues is working on this value:

private const int AUDIO_CLIP_LENGTH_SECONDS = 3600;

He mentioned:

I have changed its value to 30; and it is working.

I have also tested it on lower value around 60. I don't know how this is working on lower values.

Additionally, I am using the following documentation to implement this script:

• Audio Input | MagicLeap Developer Documentation

I am a bit confused, why crash is just related to just few params.

[kbabilinski]

Thank you for that information. I will work with the Unity SDK team to update the documentation for these parameters. I'll try to get information, but I'm happy to hear that you were able to find a workaround in the meantime.