Hi! I'm working on a project where I need to overlay a 3D model over a physical object detected in real time. When the user moves the real object the virtual object moves with it. I have looked at the magic leap documentation but it is not clear to me if this is possible to do or how it is done. The steps that would have to be performed are the real time detection and tracking of the object and the overlaying of the virtual object during the whole experience. Is this possible? Do I need to use a third party solution?
The Magic Leap SDK does not include native functionality for Object Detection and tracking, however solutions like Vuforia and Vision lib can enable this functionality.
Note, depending on your use case you may want to consider using marker tracking to track the object.
Thank you very much for your quick response, the project is for an academic environment and the goal is to use normal physical objects (usually small) to simulate a surgical training environment and move them with your hands. I think marker tracking is probably not appropriate for this use case. To understand it better, could you explain to me what data I can access from the mesh? Because looking at this example it seems that it is possible to do 3D reconstruction of objects.
I will investigate more about Vuforia and Vision Lib. Are these third party solutions compatible with Magic Jump OpenXR or not yet?
The Magic Leap 2 does not provide mesh classification but you can access the geometry of the scanned environment. The environment is returned as mesh chucks in the Unity Application and can be accessed using the ARFoundation Meshing Subsystem or Magic Leap Meshing Componenet.