Magic Leap 2 IMU data

I was unable to find any documentation on the ability to access the IMU data on the ML2. Can I read real-time data from the accelerometer, gyroscope, and magnetometer? Does this require some "developer work arounds" like the HoloLens 2 did?

3 Likes

Hi @mattycorbett For Magic Leap 2, IMU data is exposed via Android Sensor Management APIs.
You can learn more here

1 Like

Thank you! Excuse my ignorance here, but I am coming over from developing for the HoloLens and HoloLens 2. Can these APIs be used in Unity?

2 Likes

Sorry I can't answer for the Unity side of things, but I also was just looking into reading IMU data from the headset as well on the Android Native API side, so for anyone who's interested in that, this is what I found.

First, you need to modify your CMakeLists.txt to add the include directories for the NDK

include_directories(C:/Users/<your user>/AppData/Local/Android/Sdk/ndk/25.0.8775105/toolchains/llvm/prebuilt/windows-x86_64/sysroot/usr/include/android)

Then in your project, include the NDK's <sensor.h>
image

Add class member variables for an ASensorManager* and an ASensorEventQueue*

  ASensorManager* mSensorManager = nullptr;
  ASensorEventQueue* mSensorEventQueue = nullptr;

In the activity constructor I call

      // get sensor manager instance
      mSensorManager = ASensorManager_getInstanceForPackage("com.magicleap.capi.sample.camera_preview"); // your package name here (not sure why it's important, and doesn't seem to matter even if name doesn't actually exist

      // get all the available sensors
      ASensorList sensorList;
      int numSensors = ASensorManager_getSensorList(mSensorManager, &sensorList);

      // print available sensors
      for (int i = 0; i < numSensors; i++)
      {
          ASensorRef sensor = sensorList[i];
          ALOGE("%s - %s", ASensor_getName(sensor), ASensor_getStringType(sensor));
      }

      // Create an event queue that polls for sensor updates
      int looperID = 3; // random value copied from example code, not sure its significance
      mSensorEventQueue = ASensorManager_createEventQueue(mSensorManager, ALooper_forThread(), looperID,
                                                          onSensorChanged, // the static function callback that gets called on sensor update
                                                          this);    // void* user data that gets passed to callback

      // Example sensor:
      // I want to track the gyroscope data
      // Use default gyro sensor
      auto *sensor = const_cast<ASensor *>(ASensorManager_getDefaultSensor(mSensorManager, ASENSOR_TYPE_GYROSCOPE));

      // enable the gyro sensor
      auto status = ASensorEventQueue_enableSensor(mSensorEventQueue, sensor);
      // set how often to poll the sensor in micro-seconds
      int usec = 100000;
      status = ASensorEventQueue_setEventRate(mSensorEventQueue, sensor, usec);

And here is the callback called on sensor updates

    /** callback called by ASensorManager_createEventQueue() */
    static int onSensorChanged(int fd, int events, void* data)
    {
        // get the app passed as user data via void*
        CameraPreviewApp* this_app = (CameraPreviewApp*)data;

        // get all sensor events
        ASensorEvent event;
        while (ASensorEventQueue_getEvents(this_app->mSensorEventQueue, &event, 1) > 0)
        {
            // do what you will with said events
            ASensorVector gyro = event.gyro;
            ALOGE("Gyro Data: %lf, %lf, %lf", gyro.x, gyro.y, gyro.z);
        }
        return 1;
    }

And here's the list of sensors I get:

E/com.magicleap.capi.sample.camera_preview: Headset Right Accelerometer Sensor - android.sensor.accelerometer
E/com.magicleap.capi.sample.camera_preview: Headset Right Gyroscope Sensor - android.sensor.gyroscope
E/com.magicleap.capi.sample.camera_preview: Headset Left Accelerometer Sensor - android.sensor.accelerometer
E/com.magicleap.capi.sample.camera_preview: Headset Left Gyroscope Sensor - android.sensor.gyroscope
E/com.magicleap.capi.sample.camera_preview: Compute Pack Accelerometer Sensor - android.sensor.accelerometer
E/com.magicleap.capi.sample.camera_preview: Compute Pack Gyroscope Sensor - android.sensor.gyroscope
E/com.magicleap.capi.sample.camera_preview: Ambient Light Sensor - android.sensor.light
E/com.magicleap.capi.sample.camera_preview: Game Rotation Vector Sensor - android.sensor.game_rotation_vector
E/com.magicleap.capi.sample.camera_preview: Gravity Sensor - android.sensor.gravity

And some sample gyro values I get while sitting still

E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.178970, 0.111856, 0.086289
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.177905, 0.110258, 0.086822
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.176307, 0.108128, 0.087354
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.174709, 0.106530, 0.087354
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.173111, 0.104399, 0.086822
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.171513, 0.102269, 0.086289
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.170448, 0.100671, 0.085224
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.168850, 0.099073, 0.084691
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.167784, 0.097475, 0.084158
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.166719, 0.095877, 0.084158
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.165654, 0.094279, 0.084158
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.164056, 0.092681, 0.084691
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.162458, 0.091616, 0.084691
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.160860, 0.090018, 0.084158
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.159262, 0.088420, 0.083626
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.158197, 0.086822, 0.083093
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.157131, 0.085224, 0.082561
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.156066, 0.083093, 0.082028
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.155001, 0.081495, 0.082028
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.153403, 0.079897, 0.081495
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.151272, 0.078832, 0.081495
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.149142, 0.077234, 0.081495
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.147011, 0.075636, 0.081495
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.145413, 0.074038, 0.080963

I'm no expert in this by any means, and most of this I learned an hour ago by referencing this github repo that writes a shallow c++ wrapper around the api, but this should be enough to get started. Once again, sorry I can't speak to the Unity side, but this should help anyone working in the native API.

2 Likes

Does this/these features require the developer pro model? Is the answer the same with eye gaze apis?

1 Like

Yes @mattycorbett These APIs can be used in Unity.

1 Like

@mattycorbett You can work with these APIs with the Magic Leap 2 base edition. You do not need a Developer Pro license to do so. You can see a breakdown of the differences between the editions on our Where to Buy page.

1 Like

Do you have a link to the Unity APIs? Struggling to find them....

@mattycorbett At this time we have not documented this process because we use the standard Android sensor API. I recommend asking the question on Unity’s forums since it IMU data collection is not specific to our platform. You can create a native plugin to read the sensor data and pass it into your Unity application or search the Unity asset store for a precompiled version. Please let us know if you run into any specific issues.