Good morning, I really need some advice on a research project I have to do.
I need to send the acquisitions made by the Depth Camera (depth map or point clouds) to a computer connected to the same wireless network as my Magic Leap2 headset that have to analyzed the stream. I’m currently developing applications in Unity, but I can also switch to Native C.
Using the Stream Device integrated in the ML Hub I can see that it uses UDP over Wi-Fi, and rtds. I would implement these technologies in my costum application to connect to another device that will analyze the data. I’ve been looking for some ideas online, but it’s not clear to me what is actually supported.
Thank you for any advice also on different protocols or methods that allow me to make a wireless stream of depth map or point clouds.
@jasmine.santinelli Thank you for joining our discussion on the Magic Leap 2 Developer Forum. Happy to help you with this question.
The device stream only supports streaming the Mixed Reality Stream from the RGB Camera on the device. If you wish to stream the Depth Camera image, you will need to create your own application that encodes and decodes those frames.
For the encoding and decoding of frames you could using the Android NDK to encode the image into an h264 stream and then send it to the other device. However, this process is non-trivial and outside the scope of Magic Leap support as it uses the same API and process as on standard Android devices.
A slower process would be to encode the image into an image on unity and send the bytes from there. I think there is a Unity Asset on the Unity Asset Store that uses this method. However this process is less efficient and requires more processing power and I have not used the plugin myself so you may want to contact the author to verify their plugins work with Android x86_64.