MAUI project on Android using OpenXR

Hi,

I am trying to solve unexpected problems when using MAUI application on Magic Leap 2 OS 1.6.0.

I prepared a minimalist demo to check the possibility of using Magic Leap 2 for our business, but I couldn't find a working constellation for ML2.

In order to eliminate third-party inadequacy, I used two different NuGet packages for working with OpenXR: Evergine and Silk.NET.

I don't get any data from OpenXR for this manifest on Android (On Windows 11 I got 24 items):

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
	<application android:allowBackup="true" android:supportsRtl="true">
		<!-- Our activity is the built-in NativeActivity framework class.
		This will take care of integrating with our NDK code. -->
		<activity android:name="android.app.NativeActivity"
				  android:configChanges="orientation|keyboardHidden"
				  android:launchMode="singleTask"
				  android:exported="true">

			<!-- Tell NativeActivity the name of our .so -->
			<!-- Java.Lang.RuntimeException <meta-data android:name="android.app.lib_name" android:value="hand_tracking" />-->

			<!--<intent-filter>
				<action android:name="android.intent.action.MAIN" />
				--><!-- Display the application on the home launcher --><!--
				<category android:name="android.intent.category.DEFAULT" />
				<category android:name="android.intent.category.LAUNCHER" />
				--><!-- This category is defined by the OpenXR specification and is required for all OpenXR applications that run on HMD devices --><!--
				<category android:name="org.khronos.openxr.intent.category.IMMERSIVE_HMD" />
			</intent-filter>-->
		</activity>
	</application>
	<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
	<uses-permission android:name="android.permission.INTERNET" />
	<uses-permission android:name="org.khronos.openxr.permission.OPENXR_SYSTEM" />
	<uses-permission android:name="com.magicleap.permission.HAND_TRACKING" />
	<!-- Specify the requested API Level  -->
	<!--<uses-feature android:name="com.magicleap.api_level" android:version="20" />-->
	<uses-sdk />
</manifest>

I don't get a working app for this manifest:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
	<application android:allowBackup="true" android:supportsRtl="true">
		<!-- Our activity is the built-in NativeActivity framework class.
		This will take care of integrating with our NDK code. -->
		<activity android:name="android.app.NativeActivity"
				  android:configChanges="orientation|keyboardHidden"
				  android:launchMode="singleTask"
				  android:exported="true">

			<!-- Tell NativeActivity the name of our .so -->
			<meta-data android:name="android.app.lib_name" android:value="hand_tracking" />

			<intent-filter>
				<action android:name="android.intent.action.MAIN" />
				 Display the application on the home launcher 
				<category android:name="android.intent.category.DEFAULT" />
				<category android:name="android.intent.category.LAUNCHER" />
				 This category is defined by the OpenXR specification and is required for all OpenXR applications that run on HMD devices 
				<category android:name="org.khronos.openxr.intent.category.IMMERSIVE_HMD" />
			</intent-filter>
		</activity>
	</application>
	<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
	<uses-permission android:name="android.permission.INTERNET" />
	<uses-permission android:name="org.khronos.openxr.permission.OPENXR_SYSTEM" />
	<uses-permission android:name="com.magicleap.permission.HAND_TRACKING" />
	<!-- Specify the requested API Level  -->
	<uses-feature android:name="com.magicleap.api_level" android:version="20" />
	<uses-sdk />
</manifest>

In this second case I get an exception:

Java.Lang.RuntimeException
  Message=Unable to start activity ComponentInfo{com.companyname.voyager2maui/android.app.NativeActivity}: java.lang.IllegalArgumentException: Unable to find native library main using classloader: dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/com.companyname.voyager2maui-dE9idSzyNW4_Sm6Pav5rTA==/base.apk"],nativeLibraryDirectories=[/data/app/com.companyname.voyager2maui-dE9idSzyNW4_Sm6Pav5rTA==/lib/x86_64, /data/app/com.companyname.voyager2maui-dE9idSzyNW4_Sm6Pav5rTA==/base.apk!/lib/x86_64, /system/lib64, /system/product/lib64, /vendor/lib64]]]

Creating an application is simple, based on the basic MAUI application in Visual Studio 2022.
Make sure you have installed all necessary components in the installer for your Studio, such as ".NET MAUI SDK Windows" and "Android SDK setup"...
In Visual Studio, use Create new Project and search for .NET MAUI App.
Add NuGet packages to the project:
Evergine.OpenXR
OpenXR.Loader
Silk.NET.OpenXR
Silk.NET.OpenXR.Extensions.KHR
Silk.NET.OpenXR.Extensions.MAGICLEAP
Silk.NET.OpenXR.Extensions.MSFT

In Properties of the project go to the Build and select Allow Unsafe code.

In the root of the project add file "libopenxr_loader.so".
You can get this file from the web

from file hello_xr-Vulkan-release-1.1.36.apk
if you rename it to the ZIP file and get the file from archive path \lib\x86_64.

In the project in the file "MainPage.xaml.cs" change the code to:

Add:

using Evergine.Bindings.OpenXR; 
using Silk.NET.OpenXR;

Overvrite:

public partial class MainPage : ContentPage
{
	public MainPage()
	{
		InitializeComponent();
	}

	private void OnCounterClicked(object sender, EventArgs e)
	{
		(uint evergineCount, uint silkCount) = GetExtensionProperties();

		CounterBtn.Text = $"Evergine: {evergineCount}, Silk.NET: {silkCount}";

		SemanticScreenReader.Announce(CounterBtn.Text);
	}

	private (uint evergineCount, uint silkCount) GetExtensionProperties()
	{
		// Evergine:
		uint evergineCount;
		unsafe
		{
			_ = OpenXRNative.xrEnumerateInstanceExtensionProperties(null, 0, &evergineCount, null);
		}

		// Silk.NET
		XR xrApi = XR.GetApi();
		uint silkCount = 0;
		unsafe
		{
			_ = xrApi.EnumerateInstanceExtensionProperties((byte*)null, 0, &silkCount, null);
		}
		return (evergineCount, silkCount);
	}
}

Source for Android Manifest changes:

I think this website is missing information about parameter.
android:exported="true"

If you run the application on Windows 11 and click the button in the application, the name of the button will change to Evergine: 24, Silk.NET: 24.

I couldn't find a working form of the AndroidManifest.xml file, either the app runs but doesn't return any OpenXR properties, or it throws an exception.

Actually, you don't have to worry about what I describe in this problem, just create a working example application in MAUI and describe it in the documentation or put it on GitHub.

Thank you very much, Magic Leap is a very interesting device, I would recommend you to make Magic Leap 3 one that combines the advantages of Magic Leap 2 and HoloLens 2, this will give you the best device on the market for clear AR :wink:

(HoloLens 2, Vision Pro and Magic Leap 2 are on my desk.)

Hi Martin,

Welcome to the developer forum! To confirm, you should be able to run a 2d Android application built using .NET MAUI on a MagicLeap 2 device. ML2 is an AOSP (Android) 10 device with an x86 soc. Most Android applications, without dependencies on Google Play services, should be able to run if compiled for the platform without much issue.

I tried running the .net MAUI hello world app described in this tutorial and it compiled and ran without changes on my ML2-

Note, however, that the ML2 OS does not support any sort of hybrid 2d/3d app model. You wouldn't really be able to build an application that relies on emulating a 2d Android display to render content and also renders and composites stereo 3d content at the same time. The OS only supports running a single application in the foreground at a time, and this includes 2d Android apps. When you run a 2d app (non-'immersive'), you'll see content rendered on a single 3d slate with virtual buttons that is tethered to your head position-

What sort of experience were you trying to achieve with your application? Without a way to render MAUI UI components to a texture or something, it may be possible to build something modal where you switch between 2d & 3d activities, if that would work.

Best,
Adam

Hi Adam :slight_smile:

Thank you very much for your reply, I will try to explain better now what the goal is.

We made a solution that consists of a server, a Studio and an application for AR, we call it Voyager.

Voyager is an application for HoloLens 2 that connects to the server and the user at the Studio can provide remote assistance to the user wearing the HoloLens 2. Or the user can go through some educational tutorial with Voyager in the form of an interactive animation.

One of our customers successfully uses this to plan the layout of machines in large level halls.

Or if I'm bored at the office, I'll build a snowman:

We started looking at other devices to extend our solution to, in addition to the HoloLens 2. We have an Apple Vision Pro, and last week we had a MagicLeap 2 on loan from vr-expert.com.
So at this time I tried to see if we could create a MAUI application that could be part of our ecosystem.

During this time when we had the Magic Leap 2 here for a few days, I created this MAUI app with 3D content:
MAUI Bot via Evergine Vulkan on Magic Leap 2.zip (9.8 MB)

Then I tried to verify that I can access data from OpenXR on ML2, but I couldn't through any library, so I wanted to ask because I have information from vr-expert.com that OpenXR is fully supported on ML2.

An application created in MAUI is the same when built for Android as any other application created in Android Studio, for example.The question is, can an Android app on ML2 access OpenXR? What does the AndroidManifest.xml look like?

Topic number two is a spatial 3D application. Can a developer create such an immersive app for ML2? (No matter what type of dev tool I use.) I guess so, which is why I haven't looked into this topic yet...

Have a nice day,
Martin