Unity Editor version: 2022.3.12f1 ML2 OS version: 1.9.0 Unity SDK version: 2.4.0 Host OS: Windows
Error messages from logs (syntax-highlighting is supported via Markdown):
I have the following piece of code for my Unity application as shown in the screenshot below:
Everything else runs fine but when I make the call to MLCVCamera.GetFramePose, I am getting the following errors:
1- Error: MLCVCameraGetFramePose in the Magic Leap API failed. Reason: MLResult_PoseNotFound
2- Error: MLCamera.InternalGetFramePose failed to get camera frame pose. Reason: MLResult_PoseNotFound
Previously, everything was working fine. Occasionally this same API would fail but would get fine the next frame but lately its failure has started having adverse impact on the performance of my application. In effort to remedy the situation, I updated the OS of my ML2 to 1.9.0 after which this API has started to fail every single call that's been made to it resulting in a constant blank display every time I start my application.
Magic Leap keeps track of around 500ms (can be less depending on the application's performance) of historical head pose that can be looked up based on a frame's timestamp. If the timestamp of the frame is greater than 500ms the frame pose query will fail.
To make sure the camera pose is available, make sure your application runs at around 60fps.
Regarding performance impact between versions, which OS and SDK version were you using previously? I did not see any performance changes when testing our Magic Leap Camera example on the current and previous OS and SDK versions.
My initial suspicion is that the reduced performance is causing this error instead of the error impacting performance.
I unfortunately do not remember the previous OS and SDK versions. However, can you tell me how can I test if the timestamp of the frame is greater than 500ms?
To monitor performance, I am logging FPS in every Update function call in Unity. When the application starts, FPS is logged only once (value of FPS logged is 59) before a frame is detected which triggers the RawVideoFrameAvailable function that contains GetFramePose API call in my code and the call fails and keeps on failing resulting in display going blank completely.
You can convert the frame timestamp into system time var result = MLTime.ConvertMLTimeToSystemTime(mltimes[i], out long time);then compare it with the current time. You may need to consider decreasing the amount of processing that is happening in your application or moving the resource heavy task into a background thread so that the app runs properly.
--
Additionally, make sure Perception Snapshots are enabled inside your Project Settings -> XR Plugin Management -> OpenXR -> Gear icon next to MagicLeap 2 support.
Are there any specific steps that I would need to perform to reproduce the issue inside the Magic Leap Examples project?
I did strip down my application of all the processing intensive tasks such that it only now has the code from camera example code, and it is being visualized as a texture on a quad game object. I am getting 60 FPS in this stripped-down version. As soon as I include the MLCVCamera.GetFramePose API call in the RawVideoFrameAvailable function (in this stripped-down version of the application whichh is running at 60 FPS), the API starts failing and I get a blank screen.
Additionally, I was making use of examples from Unity ML SDK (as shown in the documentation here) but I noticed that it says Deprecated next to it. Could that be causing this issue? Since I am not using OpenXR I am unable to see the Perception Snapshots option in my project settings which are as follows:
Interesting, do you mind trying to use MLSDK version 1.12.0 or 2.0.0? Do you mind providing steps to reproduce this starting from the Magic Leap 1.12.0 Examples CVCamera Example?
Alright. I am getting a bit confused with all the OS and SDK versions. I tried using ML Project Setup Tool's "Set Magic Leap SDK Folder" to change the version but I am being shown v1.9.0 as my current one and there is no option for 1.12.0 in the dropdown. In Magic Leap hub under the Unity section, I have 2.4.0 versions listed and installed. Can you guide me how I can go about setting the relevant version from all these
The Unity SDK version is 1.12.0 but the MLSDK version was 1.4.1.
You can download the complete example project by selecting the examples in the Magic Leap Hub 2 or Magic Leap Hub 3 and then selecting the 1.12.0 version from the drop down.
Note that v2.0.0 is configured for OpenXR by default so I recommend testing version 1.12.0 and seeing if that version works for you.
Also, you asked about updating/downgrading your device.
If you run into issues with your hardware or end user products see the guides on our customer care page and feel free to open a ticket with them if you need any help. https://www.magicleap.care/hc/en-us
The team that operates on these forums focuses on developer related questions.
Anyway, if you get some reproduction steps I can better assist you. Otherwise, I'm not sure if the problem stems from your script or your project configuration. Additionally, you can use the latest Magic Leap Example project and use the MLCamera API with OpenXR as long as you enable perception snapshots as mentioned here: Additional Features | MagicLeap Developer Documentation
