Give us as much detail as possible regarding the issue you're experiencing:
Unity Editor version: 2022.3.42f1
ML2 OS version: 1.12.0
Unity SDK version: 2.6.0
Host OS: Windows
Hi,
My eye tracker code had stopped working since the recent update. Log is as follows:
NullReferenceException: Object reference not set to an instance of an object.
at MagicLeap.OpenXR.Features.EyeTracker.MagicLeapEyeTrackerFeature.GetStaticDataInternal () [0x00000] in <00000000000000000000000000000000>:0
at MagicLeap.OpenXR.Features.EyeTracker.MagicLeapEyeTrackerFeature.GetEyeTrackerData () [0x00000] in <00000000000000000000000000000000>:0
at EyeTracker.FixedUpdate () [0x00000] in <00000000000000000000000000000000>:0
Given that the bug comes from GetStaticDataInternal(), I believe this probably isn't about my code implementation. The code ran well on OS 1.12.0 with Unity SDK 2.5.0 and MLSDK 1.10.0 before. I also had trouble reverting my Unity project to use the old sdk, since whenever I change the Unity package version it goes to "loading assets" forever and if I shut down my Unity project and reopen, the same log would show up after deployment (even though it's showed in the project setup tool that I'm on mlsdk 1.10.0 and unity sdk 2.5.0). I have many calls to the eye tracker API in my code, but I'll attach my code here:
void Start()
{
// Request necessary permissions
Permissions.RequestPermissions(new string[]
{
Permissions.EyeTracking,
Permissions.PupilSize
}, OnPermissionGranted, OnPermissionDenied);
// InputSubsystem.Extensions.MLEyes.StartTracking()
CenterEyePositionAction = new InputAction(binding: "<EyeGaze>/pose/position");
EyeRotationAction = new InputAction(binding: "<EyeGaze>/pose/rotation");
triggerAction = new InputAction(binding: "<MagicLeapController>/trigger");
headRotationAction = new InputAction(binding: "<OpenXRHmd>/deviceRotation");
CenterEyePositionAction.Enable();
EyeRotationAction.Enable();
triggerAction.Enable();
headRotationAction.Enable();
}
void FixedUpdate()
{
// Only proceed if both permissions have been granted
if (!eyeTrackPermissionGranted || !pupilSizePermissionGranted)
return;
String row = "0";
EyeTrackerData eyeTrackerData = eyeTrackerFeature.GetEyeTrackerData();
bool hasData = eyeTrackerData.PosesData.Result == XrResult.Success;
//Obtain gaze tracking input
// vergence is gone in Unity SDK 2.5.0
// bool hasData = _eyeTrackingDevice.TryGetFeatureValue(EyeTrackerUsages.vergencePosition, out Vector3 vergencePosition) &
// _eyeTrackingDevice.TryGetFeatureValue(EyeTrackerUsages.vergenceRotation, out Quaternion vergenceRotation);
if (hasData)
{
fixationPoint = eyeTrackerData.PosesData.FixationPose.Pose.position;
}
else
{
fixationPoint = Vector3.zero;
}
if (eyeTrackerFeature != null && eyeTrackerFeature.enabled)
{
// Retrieve all eye-tracking data
long currentTime = eyeTrackerData.GazeBehaviorData.Time / 1000000;
if (prevGazeTime == currentTime)
{
// Debug.Log("Duplicate gaze time. Skipping frame: " + currentTime);
return;
}
prevGazeTime = currentTime;
row += "," + currentTime.ToString();
// Debug.Log($"Gaze Time: {eyeTrackerData.PupilData[0].Time}");
// Log Pupil Data
foreach (var pupilData in eyeTrackerData.PupilData)
{
// Debug.Log($"Pupil Data - Eye: {pupilData.Eye}, Diameter: {pupilData.PupilDiameter} meters");
row += "," + pupilData.PupilDiameter.ToString("F4");
}
// fixation point
// Vector3 centerEyePosition = CenterEyePositionAction.ReadValue<Vector3>();
// Quaternion centerEyeRotation = EyeRotationAction.ReadValue<Quaternion>();
// or, use Eye Tracker instead of Input System
Vector3 centerEyePosition = eyeTrackerData.PosesData.GazePose.Pose.position;
Quaternion centerEyeRotation = eyeTrackerData.PosesData.GazePose.Pose.rotation;
Vector3 centerEyeDirection = centerEyeRotation * Vector3.forward;
Quaternion headRotation = headRotationAction.ReadValue<Quaternion>();
row += "," + centerEyePosition.ToString("F3").Replace(",", "|");
row += "," + centerEyeRotation.ToString("F3").Replace(",", "|");
row += "," + centerEyeDirection.ToString("F3").Replace(",", "|");
row += "," + fixationPoint.ToString("F3").Replace(",", "|");
row += "," + headRotation.ToString("F3").Replace(",", "|")
// Debug.Log($"Fixation Point: {fixationPoint}");
// Debug.Log("Center Eye Position: " + centerEyePosition);
// Debug.Log("Center Eye Rotation: " + centerEyeRotation);
// Debug.Log("Center Eye Rotation Direction: " + centerEyeDirection);
if (fixationPoint == Vector3.zero)
{
// Debug.Log("Vergence Point not found. Using Center Eye Position and Rotation.");
fixationPoint = centerEyePosition + centerEyeRotation * Vector3.forward * 1f;
}
// cube at fixation point
if (showFixationCube)
{
if (fixationCube == null)
{
fixationCube = GameObject.CreatePrimitive(PrimitiveType.Cube);
fixationCube.AddComponent<Renderer>();
fixationCube.GetComponent<Renderer>().material = new Material(Shader.Find("Universal Render Pipeline/Lit"));
fixationCube.transform.localScale = new Vector3(0.025f, 0.025f, 0.025f);
}
fixationCube.transform.position = fixationPoint;
}
// Log Gaze Behavior Data
GazeBehavior gazeBehavior = eyeTrackerData.GazeBehaviorData;
// Debug.Log($"Gaze Behavior - Type: {gazeBehavior.GazeBehaviorType}, Duration: {gazeBehavior.Duration}, Amplitude: {gazeBehavior.MetaData.Amplitude}, Direction: {gazeBehavior.MetaData.Direction}, Velocity: {gazeBehavior.MetaData.Velocity}");
row += "," + gazeBehavior.MetaData.Amplitude.ToString("F3");
row += "," + gazeBehavior.MetaData.Direction.ToString("F3");
row += "," + gazeBehavior.MetaData.Velocity.ToString("F3");
row += "," + gazeBehavior.GazeBehaviorType.ToString();
row += "," + gazeBehavior.Time.ToString();
row += "," + gazeBehavior.OnsetTime.ToString();
row += "," + gazeBehavior.Duration.ToString();
// Log Geometric Data
foreach (var geometricData in eyeTrackerData.GeometricData)
{
// Debug.Log($"Geometric Data - Eye: {geometricData.Eye}, Openness: {geometricData.EyeOpenness}, Position: {geometricData.EyeInSkullPosition}");
row += "," + geometricData.EyeOpenness.ToString("F3");
}
// log gaze target data
var ray = new Ray(centerEyePosition, centerEyeDirection);
if (Physics.Raycast(ray, out RaycastHit hit, 15f, ~(1 << LayerMask.NameToLayer("Ignore Raycast"))))
{
row += "," + hit.transform.name;
}
else
{
row += ",None";
}
row += "," + triggerAction.ReadValue<float>().ToString("F3");
if (showFixationCube)
{
switch (gazeBehavior.GazeBehaviorType)
{
case GazeBehaviorType.Fixation:
fixationCube.GetComponent<Renderer>().material.color = Color.yellow;
break;
case GazeBehaviorType.Saccade:
fixationCube.GetComponent<Renderer>().material.color = Color.red;
break;
case GazeBehaviorType.Pursuit:
fixationCube.GetComponent<Renderer>().material.color = Color.green;
break;
case GazeBehaviorType.Blink:
fixationCube.GetComponent<Renderer>().material.color = Color.blue;
break;
case GazeBehaviorType.EyesClosed:
fixationCube.GetComponent<Renderer>().material.color = Color.gray;
break;
}
}
}
}
I am also a bit confused by the release note here - in the release note of 2.6.0, it says "Unity Developers that used the experimental Eye Tracker OpenXR Feature should update their Unity SDK to 2.5.0. The new 1.11.0 OS is incompatible with the previous version of the API." Is this a typo or should I stick with 2.5.0 and 1.11.0 OS?
If there's no quick fix to this, could you please advise on how to properly revert my Unity project?
Thanks in advance for your help!