Unity Editor version: 2022.2.0b9.3080
ML2 OS version: B3E.220818.12-R.046-R.417
MLSDK version: 1.0.0
Host OS: WINDOWS
Agora SDK 3.7.1
No errors - The service runs correctly for ~5 seconds (green microphone icon visible in the left top corner of FOV).
During this time, audio is successfully transmitted to other users. After approximately 5 seconds, the microphone disappears and audio data is no longer transmitted from the ML2, but audio is still received from other users.
@rick any ideas on what might be happening here?
@drew do you see any thing suspicious from the adb log? If you grep for Unity, is there any error happening in the Unity application layer, like an exception?
I'll check again today, I didn't notice anything in logcat for the system/Unity
Ah - I do have some messages here - they snuck by me.
Nothing in unity, but this looks interesting:
10-20 21:30:12.941 22539 22789 I libOpenSLES: Emulating old channel mask behavior (ignoring positional mask 0x3, using default mask 0x3 based on channel count of 2)
10-20 21:30:12.941 22539 22789 E AudioTrack: AudioTrack Constructor 2 not supported
10-20 21:30:12.941 3439 22056 E audiorenderservice: audio_render.cc:225: Track listing (135095420578592,10100,22539) not found
10-20 21:30:12.942 3439 22056 E audiorenderservice: audio_render_service.cc:332: MLAudioDestroySound: [FAILED (-1643053055) (MLAudioResult_HandleNotFound)]
10-20 21:30:12.942 22539 22789 E AudioTrack: Failed to destroy audio output: -1643053055
10-20 21:30:12.942 22539 22789 E libOpenSLES: AudioTrack::initCheck status 1519250568
10-20 21:30:12.942 22539 22789 W libOpenSLES: Leaving Object::Realize (SL_RESULT_CONTENT_UNSUPPORTED)
10-20 21:30:12.942 22539 22789 W libOpenSLES: Conversion from OpenSL ES positional channel mask 0x4 to Android mask 0 loses channels
10-20 21:30:12.942 22539 22789 I libOpenSLES: Emulating old channel mask behavior (ignoring positional mask 0x4, using default mask 0x10 based on channel count of 1)
10-20 21:30:12.942 22539 22789 I AudioRecord: Format:
10-20 21:30:12.942 22539 22789 I AudioRecord: samples_per_second=48000
10-20 21:30:12.942 22539 22789 I AudioRecord: channel_count=1
10-20 21:30:12.942 22539 22789 I AudioRecord: bits_per_sample=16
10-20 21:30:12.942 22539 22789 I AudioRecord: valid_bits_per_sample=16
10-20 21:30:12.942 22539 22789 I AudioRecord: sample_format=0
10-20 21:30:12.942 22539 22789 I AudioRecord: Buffer Size: 1920 bytes (960 frames)
10-20 21:30:12.943 3423 3423 I audiocaptureservice: audio_capture.cc:198: New stream listing (30,10100,22539)
10-20 21:30:12.944 3423 4960 I audiocaptureservice: mea_device.cc:118: MEA: attempting to grab mutex for open
10-20 21:30:12.944 3423 4960 I audiocaptureservice: mea_device.cc:123: MEA: attempting to open mea device 2
10-20 21:30:12.944 4379 4412 I AudioProxyService: setCaptureAudioIndication: isRecording = true
10-20 21:30:12.945 3423 4960 I audiocaptureservice: mea_device.cc:133: MEA: successfully opened mea device 2 with pointer 0x78d6ba891c10
10-20 21:30:12.965 4403 4896 W MagicleapGodotApp: Loading resource: res://Indicators/Indicators.tscn
10-20 21:30:12.966 4403 4896 W MagicleapGodotApp: Loading resource: res://Indicators/textures/circle_white_128px.png
10-20 21:30:12.967 4403 4896 W MagicleapGodotApp: Loading resource: res://Indicators/animation/indicator_dual.anim
10-20 21:30:12.968 4403 4896 W MagicleapGodotApp: Loading resource: res://Indicators/animation/indicator_passive.anim
10-20 21:30:12.969 4403 4896 W MagicleapGodotApp: Loading resource: res://Indicators/animation/indicator_triple.anim
10-20 21:30:12.970 4403 4896 W MagicleapGodotApp: Loading resource: res://Indicators/animation/indicator_single.anim
10-20 21:30:12.970 4403 4896 W MagicleapGodotApp: Loading resource: res://Indicators/Indicators.gd
10-20 21:30:13.707 4379 4412 I AudioProxyService: setCaptureAudioIndication: isRecording = false
10-20 21:30:13.708 3423 4960 I audiocaptureservice: mea_device.cc:150: MEA: attempting to close mea device 2 with pointer 0x78d6ba891c10
10-20 21:30:13.708 3423 4960 I audiocaptureservice: mea_device.cc:152: MEA: returned from mea device close (success unknown)
10-20 21:30:13.711 22539 22789 E libOpenSLES: Object::Destroy(0x7add55408600) for OutputMix ignored; 1 players attached
10-20 21:30:13.711 22539 22789 E libOpenSLES: Object::Destroy(0x7add55408600) not allowed
10-20 21:30:13.711 22539 22789 E libOpenSLES: Object::Destroy(0x7add5a5fd000) for engine ignored; 2 total active objects
10-20 21:30:13.711 22539 22789 E libOpenSLES: Object::Destroy(0x7add5a5fd000) for engine ignored; active object ID 2 at 0x7add55408600
10-20 21:30:13.711 22539 22789 E libOpenSLES: Object::Destroy(0x7add5a5fd000) for engine ignored; active object ID 3 at 0x7add3eb34c00
10-20 21:30:13.720 22539 22889 I AudioTrack: Format:
10-20 21:30:13.720 22539 22889 I AudioTrack: samples_per_second=48000
10-20 21:30:13.720 22539 22889 I AudioTrack: channel_count=2
10-20 21:30:13.720 22539 22889 I AudioTrack: bits_per_sample=16
10-20 21:30:13.720 22539 22889 I AudioTrack: valid_bits_per_sample=16
10-20 21:30:13.720 22539 22889 I AudioTrack: sample_format=0
10-20 21:30:13.720 22539 22889 I AudioTrack: channel_format=0
10-20 21:30:13.721 22539 22889 W AudioTrack: Use of stream types is deprecated for operations other than volume control
10-20 21:30:13.721 22539 22889 W AudioTrack: See the documentation of AudioTrack() for what to use instead with android.media.AudioAttributes to qualify your playback use case
10-20 21:30:13.722 22539 22539 V MediaRouter: Selecting route: RouteInfo{ name=Phone, description=null, status=null, category=RouteCategory{ name=System types=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO groupable=false }, supportedTypes=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO , presentationDisplay=null }
10-20 21:30:13.725 22539 22889 E AudioRecord-JNI: Error creating AudioRecord instance: initialization check failed with status -1.
10-20 21:30:13.725 3423 3423 E audiocaptureservice: audio_capture.cc:127: Stream listing (1065353216,10100,22539) not found
10-20 21:30:13.725 3423 3423 E audiocaptureservice: audio_capture_service.cc:440: MLAudioStopInput: [FAILED (-1643053055) (MLAudioResult_HandleNotFound)]
10-20 21:30:13.725 3423 3423 E audiocaptureservice: audio_capture.cc:127: Stream listing (1065353216,10100,22539) not found
10-20 21:30:13.725 3423 3423 E audiocaptureservice: audio_capture_service.cc:399: MLAudioDestroyInput: [FAILED (-1643053055) (MLAudioResult_HandleNotFound)]
10-20 21:30:13.725 22539 22889 E AudioRecord: Failed to destroy audio input: -1643053055
10-20 21:30:13.725 22539 22889 E android.media.AudioRecord: Error code -20 when initializing native AudioRecord object.
Hi Drew, I got no clue from these messages. Let's go through how you integrated Agora. Did you use the AgoraController prefab that is included in the SDK? Did you try out the demo and see the same audio issue happening there?
@drew , can you confirm if you tested with the demo we included. Also, the base SDK for the plugin is actually 3.7.0.3. When you said 3.7.1, did you mean you download the Unity SDK and use it directly? If so, that could be the gap here. We have a dedicated SDK bundle built for the ML2. You may find it here.
The custom audio source scripts work just fine, thanks.
We have been internally doing the same thing (using a custom audio source) until recently when I was told this was no longer necessary and we would no longer need to do so.
Again, this works for us, but I do have a question:
The quality is fine, but the transmission latency is more noticeable than on other devices (like Oculus Quest 2). Should we expect relative parity between the custom audio source implementation vs what I'm assuming is a native implementation?
Edit: Quick test gives us ~1.5 seconds on Quest 2 -> Desktop vs ~5 seconds ML2->Desktop
Hi Drew, we will look into the latency issue and see if there is optimization that can be done. Also, we will check if the audio can be done without custom audio source now. Do you have a reference to that knowledge?