I was unable to find any documentation on the ability to access the IMU data on the ML2. Can I read real-time data from the accelerometer, gyroscope, and magnetometer? Does this require some "developer work arounds" like the HoloLens 2 did?
3 Likes
Hi @mattycorbett For Magic Leap 2, IMU data is exposed via Android Sensor Management APIs.
You can learn more here
1 Like
Thank you! Excuse my ignorance here, but I am coming over from developing for the HoloLens and HoloLens 2. Can these APIs be used in Unity?
2 Likes
Sorry I can't answer for the Unity side of things, but I also was just looking into reading IMU data from the headset as well on the Android Native API side, so for anyone who's interested in that, this is what I found.
First, you need to modify your CMakeLists.txt to add the include directories for the NDK
include_directories(C:/Users/<your user>/AppData/Local/Android/Sdk/ndk/25.0.8775105/toolchains/llvm/prebuilt/windows-x86_64/sysroot/usr/include/android)
Then in your project, include the NDK's <sensor.h>
Add class member variables for an ASensorManager* and an ASensorEventQueue*
ASensorManager* mSensorManager = nullptr;
ASensorEventQueue* mSensorEventQueue = nullptr;
In the activity constructor I call
// get sensor manager instance
mSensorManager = ASensorManager_getInstanceForPackage("com.magicleap.capi.sample.camera_preview"); // your package name here (not sure why it's important, and doesn't seem to matter even if name doesn't actually exist
// get all the available sensors
ASensorList sensorList;
int numSensors = ASensorManager_getSensorList(mSensorManager, &sensorList);
// print available sensors
for (int i = 0; i < numSensors; i++)
{
ASensorRef sensor = sensorList[i];
ALOGE("%s - %s", ASensor_getName(sensor), ASensor_getStringType(sensor));
}
// Create an event queue that polls for sensor updates
int looperID = 3; // random value copied from example code, not sure its significance
mSensorEventQueue = ASensorManager_createEventQueue(mSensorManager, ALooper_forThread(), looperID,
onSensorChanged, // the static function callback that gets called on sensor update
this); // void* user data that gets passed to callback
// Example sensor:
// I want to track the gyroscope data
// Use default gyro sensor
auto *sensor = const_cast<ASensor *>(ASensorManager_getDefaultSensor(mSensorManager, ASENSOR_TYPE_GYROSCOPE));
// enable the gyro sensor
auto status = ASensorEventQueue_enableSensor(mSensorEventQueue, sensor);
// set how often to poll the sensor in micro-seconds
int usec = 100000;
status = ASensorEventQueue_setEventRate(mSensorEventQueue, sensor, usec);
And here is the callback called on sensor updates
/** callback called by ASensorManager_createEventQueue() */
static int onSensorChanged(int fd, int events, void* data)
{
// get the app passed as user data via void*
CameraPreviewApp* this_app = (CameraPreviewApp*)data;
// get all sensor events
ASensorEvent event;
while (ASensorEventQueue_getEvents(this_app->mSensorEventQueue, &event, 1) > 0)
{
// do what you will with said events
ASensorVector gyro = event.gyro;
ALOGE("Gyro Data: %lf, %lf, %lf", gyro.x, gyro.y, gyro.z);
}
return 1;
}
And here's the list of sensors I get:
E/com.magicleap.capi.sample.camera_preview: Headset Right Accelerometer Sensor - android.sensor.accelerometer
E/com.magicleap.capi.sample.camera_preview: Headset Right Gyroscope Sensor - android.sensor.gyroscope
E/com.magicleap.capi.sample.camera_preview: Headset Left Accelerometer Sensor - android.sensor.accelerometer
E/com.magicleap.capi.sample.camera_preview: Headset Left Gyroscope Sensor - android.sensor.gyroscope
E/com.magicleap.capi.sample.camera_preview: Compute Pack Accelerometer Sensor - android.sensor.accelerometer
E/com.magicleap.capi.sample.camera_preview: Compute Pack Gyroscope Sensor - android.sensor.gyroscope
E/com.magicleap.capi.sample.camera_preview: Ambient Light Sensor - android.sensor.light
E/com.magicleap.capi.sample.camera_preview: Game Rotation Vector Sensor - android.sensor.game_rotation_vector
E/com.magicleap.capi.sample.camera_preview: Gravity Sensor - android.sensor.gravity
And some sample gyro values I get while sitting still
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.178970, 0.111856, 0.086289
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.177905, 0.110258, 0.086822
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.176307, 0.108128, 0.087354
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.174709, 0.106530, 0.087354
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.173111, 0.104399, 0.086822
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.171513, 0.102269, 0.086289
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.170448, 0.100671, 0.085224
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.168850, 0.099073, 0.084691
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.167784, 0.097475, 0.084158
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.166719, 0.095877, 0.084158
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.165654, 0.094279, 0.084158
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.164056, 0.092681, 0.084691
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.162458, 0.091616, 0.084691
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.160860, 0.090018, 0.084158
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.159262, 0.088420, 0.083626
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.158197, 0.086822, 0.083093
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.157131, 0.085224, 0.082561
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.156066, 0.083093, 0.082028
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.155001, 0.081495, 0.082028
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.153403, 0.079897, 0.081495
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.151272, 0.078832, 0.081495
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.149142, 0.077234, 0.081495
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.147011, 0.075636, 0.081495
E/com.magicleap.capi.sample.camera_preview: Gyro Data: -0.145413, 0.074038, 0.080963
I'm no expert in this by any means, and most of this I learned an hour ago by referencing this github repo that writes a shallow c++ wrapper around the api, but this should be enough to get started. Once again, sorry I can't speak to the Unity side, but this should help anyone working in the native API.
2 Likes
Does this/these features require the developer pro model? Is the answer the same with eye gaze apis?
1 Like
@mattycorbett You can work with these APIs with the Magic Leap 2 base edition. You do not need a Developer Pro license to do so. You can see a breakdown of the differences between the editions on our Where to Buy page.
1 Like
Do you have a link to the Unity APIs? Struggling to find them....
@mattycorbett At this time we have not documented this process because we use the standard Android sensor API. I recommend asking the question on Unity’s forums since it IMU data collection is not specific to our platform. You can create a native plugin to read the sensor data and pass it into your Unity application or search the Unity asset store for a precompiled version. Please let us know if you run into any specific issues.