MediaPipe on ML2

Give us as much detail as possible regarding the issue you're experiencing.

Unity Editor version: 2022.2.19
ML2 OS version:1.3.0-dev2
MLSDK version:1.8.0
Host OS: (Windows/MacOS) Microsoft Win 11

Error messages from logs (syntax-highlighting is supported via Markdown):

I am trying to use MediaPipe via MediaPipeForUnity (Installation Guide · homuler/MediaPipeUnityPlugin Wiki · GitHub) . I am building the plugins as directed (GitHub - homuler/MediaPipeUnityPlugin: Unity plugin to run MediaPipe). The plugins current reside in the project's Packages/MediaPipeUnityPlugin/Runtime/Plugin/Android folder, which contains a file called "mediapipe_android.aar" The project works just fine in the Unity Editor, but when I deploy I get the error above about not finding the binaries. I am building the binaries using the "fat" options, instead of the ARM64 option. Still no luck. Has anyone else deployed this before? Why is the ML2 not finding the binaries at runtime?

Please try to recompile the plugin for x86_64 before deploying it to the device. The .aar file should have an x86_64 folder with the associated .so files when compiled correctly

@kbabilinski. OK. Thank you. I see the .so, .dll, and .dylib in another folder higher in the directory the plugin built. Is that OK? Attached is a pic of the higher directory is attached. The only options or the build are ARM64, ARM7, or FAT. I assume FAT is the correct option?

image

You will need to change the build script and edit the plugin for it to run on Magic Leap 2. After changing the mediapipe_android.aar file into mediapipe_android.zip you can see that there is no x86_64 version in the precompiled version of the plugin.

You should modify the build scripts to include an x86_64 variant:

After speaking with the code owner, I have modified the .bazelrc file on lines 62 and 63 to have x86_64 on the line. The new script produces the .aar file that has the x86_64 folder, but the problem still persists. Is the .aar the only library file I need to have?

Can you verify that the contents of the x86_64 file are the same as the arm64?

They are. I actually have had some success and have the libraries loaded. However the plugin still gives me errors (below). The code base author is telling me its an issue I can fix with the config, but I cant find the glogs (post). Is there a standard place I should look for these? See errors below.

Turns out that it was an implementation issue. After working with the code base author, I was able to get the chipset buildcorrect. You can find how I did that on the links above. The rest was re-working the sample implementations and added additional image capture classes using the MLCamera APIs.

1 Like

Thank you for sharing information about your solution. :slightly_smiling_face: