Give us as much detail as possible regarding the issue you're experiencing.
ML2 OS version: MLSDK version: Host OS: Windows
I tried using a tool “llama.cop” from GitHub to employ LLM like Llama2 in Magic Leap 2, but I don’t know how to achieve this. My I dear is to make llama.cop a kind of library which I can include it in main.cpp in my project in Android Studio, like you did in the ML c-api samples. However, I don’t know whether my thoughts are right and how to invoke such tool like llama.cpp that can run LLM. If this idea is feasible, please teach me how to do.
Another idea is to use libtorch, which I haven’t tried yet. Please let me know if this way is feasible and which way is better.
Sounds like an interesting project! While integrating LLMs like Llama2 into Magic Leap 2 falls outside the direct support we offer on this forum, I can guide you to some helpful resources that might help:
Llama.cpp Integration:
For understanding how to include external libraries in Android Studio, check out this official Android Developer Guide.
To learn more about using Llama.cpp, visit their GitHub repository for examples.
Thank you very much. Now I am trying to run GPT-2 on magic leap with the ggml library, which may be easier to realize than with libtorch. By now, I have succeeded in running ggml examples on my Windows computer. However, I don't know how to link all the libraries and the header files to my magic leap application project on Android Studio. Could you please offer me some guidance on how to link a whole of library to my project so that I can do something like #include "ggml.h" in my main.cpp?
It sounds like you're making good progress. To link the ggml library and header files to your Magic Leap application project in Android Studio, I recommend checking out this tutorial:
Thank you for your prompt help. My project is based on a c-api sample, which means I just changed main.cpp, while other files remain still, and it works properly. Now I want to link .dll and .lib libraries, and I stored them under User\MagicLeap\mlsdk\v1.7.0\lib\win. Then I copied every needed .h file to the src/cpp with my main.cpp file. However, it didn't work. Could you please help me to figure out where I should store the .dll and .lib files?
/Below is my CMakeLists.txt./
cmake_minimum_required(VERSION 3.22.1)
For Android projects, you would use .so files rather than .dll or .lib. Here is a useful guide on how to manage native dependencies in Android projects: Using C/C++ libraries in Android Studio.
Your CMakeLists.txt needs to point to the correct paths for your libraries and you may need to adjust your target_link_libraries and include directory settings. Refer to this guide on CMake with Android NDK.
Since your question touches on broader development practices, I suggest posting and looking at questions listed on Stack Overflow. The community there is quite helpful with issues like these. You may also want to look at LLMs that can help you with specific scripts.
You may want to try building a standard Android app that targets Android 10 and the x86_64 architecture. The Magic Leap 2 will be able to run the app as long as it does not have dependences on google paly services and display it as a 2D panel.
Then you can test your implementation without worrying about Magic Leap Specific configurations. Once you have the app created, you can work on creating a library file that can be integrated into the C++ project , or better yet a native Android OpenXR app:
I see. But mask do have ml folder containing .so files and also win folder containing .dll and .lib files, such as android.dll. So I guess maybe c-api samples also used files in win folder, which means magic leap can use .dll and .lib files. I don’t know whether this idea is right.
| - |
Date | 07/22/2024 23:20 |
To | 1776042905317760429053@163.com |
Subject | [Magic Leap 2 Developer Forums] [Native Development] About developing an App on Android Studio that can interact with a LLM locally |
For Android projects, you would use .so files rather than .dll or .lib. Here is a useful guide on how to manage native dependencies in Android projects: Using C/C++ libraries in Android Studio.
Your CMakeLists.txt needs to point to the correct paths for your libraries and you may need to adjust your target_link_libraries and include directory settings. Refer to this guide on CMake with Android NDK.
Since your question touches on broader development practices, I suggest posting and looking at questions listed on Stack Overflow. The community there is quite helpful with issues like these. You may also want to look at LLMs that can help you with specific scripts.
Thank you so much.
Could you tell me how to check the Current working directory of an application. I need to load the model when running my project, so I may need to store the model file in the Current directory.
|