Guidance for MR behaviour experiment with static object positioning and collecting data

General
, ,
1

Guidance for MR behavior experiment with static object positioning and collecting data

Hi, I am currently looking for guidance in designing an AR/MR behavioral experiment where I need to track participants' behavior while they interact and navigate in a mixed reality space physically. Here are the requirements and the details of my project:

  1. I need to place/anchor objects are fixed locations.
  2. The experiment will be repeated over time among many participants at different times, so I need functionality to always spawn the objects at the designated fixed locations every time I reopen my AR app
  3. I'm planning all my developments on Unreal.

Here are my doubts based on my current level of understanding:

  1. I was planning to use spatial anchors but I am not sure how to exactly display that. I read that I can scan my environment at export it as a .gbl file. How can I use that in Unreal to anchor objects in my real space?
  2. Secondly how can I make sure in my unreal project, that objects are spawned based on the spatial anchor and not relative to where I turn on the headset? Do I need to use a QR to initiate spatial tracking?
  3. Lastly since I need to collect positional and orientation data of user's behavior, how can I store this data on Magicleap, is it the same as coding for a
    normal unreal project or do I need to use some special file io for collecting and saving data while the application is running on ML?
  4. Is there a way to directly collect eye tracking data on ML2, how can I enable this on my app using Unreal?

Lastly, if you think there is an alternative approach that I'm not aware of, feel free to suggest I will be so happy for any input.

2

Hello,

Welcome to the forums! You're correct that spatial anchors and marker tracking are the solutions that are provided out of the box to anchor static content in your space. To clarify, spatial anchors are an alternative to using QR codes (or other markers) to anchor content in your environment. Determining which approach is more appropriate would depend on the requirements of your application.

Spatial anchors are reference frames of interest within the bounds of a spatial map. (called a "Space" on MagicLeap) Basically, you would want to create one at each location where you would want to place content in your environment. (you can think of them as creating a virtual/invisible qr code) The main benefit of using spatial anchors is that you do not have to physically place markers in your space. Additionally, they can be located without the user necessarily having to specifically look where the markers would be placed. Each anchor will have a unique id associated with it that you can use to differentiate between them. However, there are trade-offs- In order to use them, you have to create a map of your space ahead of time using the Spaces app. Then, in order to locate them at runtime, the device must be able to 'localize' itself in the space. Localization performance can vary depending on the quality of the map, the lighting conditions in your space, etc. Spatial anchors may also not be appropriate for accurately placing content very far away from the user.

As you mentioned, when you create and map a Space, a static 3d mesh is output as a glb file. You don't have to use that mesh at all to place content unless you have a specific use case for it. Once you have created the spatial anchors that you need to track, you don't need to do anything with the mesh in order for the system to recall and relocate them. That said, the mesh potentially could be useful depending on what you're trying to do. It's a 3d model of your space. You could use it for intersection testing (collisions / physics) in your environment, although, depending on your requirements, the live plane finding or spatial meshing systems may be more appropriate.

QR codes (and other markers) are detected and tracked at runtime. I suspect that if you're trying to build a repeatable experiment, then marker tracking may be more appropriate. Advantages of using QR codes is that you do not need to scan your space ahead of time. They're simple to place. Tracking performance at runtime will be determined by visibility of the markers, their size, and what type of markers you use. Generally, if you do not need to encode data in the maker, other than an id, we would recommend using Aruco codes for better tracking performance.

Either approach is possible using Unreal. What is the size of your space?

As for your other questions, file io restrictions will be similar to building an app for an Android device. You would need to request permissions as appropriate depending on where you want to save the file.

Eye tracking data can be accessed in Unreal using Epic's own eye tracking plugin. We have a little bit of documentation on it here- Eye Tracking | MagicLeap Developer Documentation

Best,
Adam

3

Dear Adam, thank you so much for your descriptive reply. I have some follow-up questions.

  1. First of all I don't want to use QR since I don't want objects to pop up when users only go close enough to the QR but I want objects to be present smoothly throughout the scene as the space of my experiment is larger (~100 sq. m). Prior scanning the hall is not a problem for me, since the experiment space remains the same;)
  2. You said I don't need the .glb model, then how can I position the objects at the specific anchors, do I need to use the unique IDs? Is there an example script or workflow for doing that on Unreal?
  3. Then about saving the position/orientation tracking and eye tracking data, I need to save it in Magicleap only, what kind of permission I need?
  4. Also I saw there is much more documentation available for Unity on spatial anchoring, do you recommend I change the platform or would u suggest me stay in Unreal?
4

Hi @abhiazadz

  1. Have you taken a look at the Unreal Example Project? I think it could help clear things up.

  2. You can create spatial anchors which will have a unique ID. Once created, you can either visualize them inside of your app or inside of the Spaces App. You will need to know the anchors Unique ID if you want to attach content to it. After querying the anchors, you can obtain each anchors position, and then position the content accordingly.

  3. You simply need the Android read/write permissions to save files.(android.permission.WRITE_EXTERNAL_STORAGE)

1 Like