MLWebViewScreenBehavior does not render WebView (OpenXR, MLSDK 2.3)

[crucial-sailor]

Dear Magic Leap Team,

I'm trying to make the MLWebView work in our OpenXR (MLSDK 2.3) unity project. It seems that either the deprecated MLWebView does not work with OpenXR or i might have missed some setup steps.

These are the versions i used:
Unity Editor version: 2022.3.19f1
ML2 OS version: 1.9
Unity SDK version: 2.3
Host OS: Windows

Unfortunately, I didn't find any WebView Example project which is using the OpenXR XR Provider. The only example project i found was for MLSDK version 1.12 which was unfortunately not using OpenXR, so i took that as a starting point.

My approach was the following:
I tried to replicate minimal gameobjects and scripts needed from the 1.12. example project, because this overview states that i can still use the deprecated MLSDK for WebViews.

I set up the MLWebViewScreenBehaviour exactly like in the 1.12. example project as you can see in the images below. Permissions, Manifest and other project settings should also be correct.

My Problem now is that the MLWebViewScreenBehaviour.WebView stays null after CreateWebViewWindow() was called which probably is also the reason why the WebView does not render and stays white/default material.

WebViewML script:
This script simply tries to navigate to the homeUrl which is just "https://google.de/". It is a very shortened and slightly modified version of the WebViewExample script from the 1.12. example project.
Notice that I also tried to wait for the webview to become initialized, but that still didn't work.

public class WebViewML : MonoBehaviour
{
    private event Action OnSetupComplete;

    [SerializeField]
    private MLWebViewScreenBehavior _mlWebViewScreenBehavior;

    [SerializeField] private string _homeUrl = "https://google.de/";

    private CancellationTokenSource _cancellationTokenSource;

    private void OnEnable()
    {
        OnSetupComplete += HandleSetupCompleted;
    }

    private void OnDisable()
    {
        OnSetupComplete -= HandleSetupCompleted;
    }

    private void Start()
    {
        try
        {
            if (Permissions.CheckPermission(Permissions.WebView))
            {
                Debug.Log("WEBVIEW: WebView Permission is granted.");
            }
            else
            {
                Debug.LogError("WEBVIEW: WebView Permission is missing.");
            }

            if (_mlWebViewScreenBehavior.CreateWebViewWindow())
            {
                Debug.Log("WEBVIEW: Successfully created web view window");

                _cancellationTokenSource = new CancellationTokenSource();

                StartSetupTask(_cancellationTokenSource.Token).Forget(ex => UnityEngine.Debug.LogException(ex));
            }
            else
            {
                Debug.LogError("WEBVIEW: Failed to create web view window");
            }
        }
        catch (Exception ex)
        {
            Debug.LogError($"WEBVIEW: {ex.Message}");
        }
    }

    private async UniTask StartSetupTask(CancellationToken cancellationToken)
    {
        while (_mlWebViewScreenBehavior.WebView == null && !cancellationToken.IsCancellationRequested)
        {
            // ERROR HERE
            Debug.LogError("WEBVIEW: WebView still null.");
            await UniTask.WaitForSeconds(1, cancellationToken: cancellationToken);
        }

        Debug.LogError("WEBVIEW: Setup completed.");
        OnSetupComplete?.Invoke();
    }

    private void HandleSetupCompleted()
    {
        Debug.Log("WEBVIEW: Handle Setup Completed");

        _mlWebViewScreenBehavior.WebView.OnServiceConnected += HandleOnServiceConnected;
        _mlWebViewScreenBehavior.WebView.OnServiceDisconnected += HandleOnServiceDisconnected;
        _mlWebViewScreenBehavior.WebView.OnServiceFailed += HandleOnServiceFailed;

        HomePage();
    }

    private void HandleOnServiceFailed(UnityEngine.XR.MagicLeap.MLWebView webView, UnityEngine.XR.MagicLeap.MLResult result)
    {
        Debug.Log("WEBVIEW: OnServiceFailed");
    }

    private void HandleOnServiceDisconnected(UnityEngine.XR.MagicLeap.MLWebView webView)
    {
        Debug.Log("WEBVIEW: OnServiceDisconnected");
    }

    private void HandleOnServiceConnected(UnityEngine.XR.MagicLeap.MLWebView webView)
    {
        Debug.Log("WEBVIEW: OnServiceConnected");
    }

    /// <summary>
    /// Load the home page.
    /// </summary>
    public void HomePage()
    {
        GoToUrl(_homeUrl);
    }

    /// <summary>
    /// Change current web page to the provided Url address.
    /// </summary>
    public void GoToUrl(String url)
    {
        if (_mlWebViewScreenBehavior.WebView != null)
        {
            if (!_mlWebViewScreenBehavior.WebView.GoTo(url).IsOk)
            {
                Debug.LogError("WEBVIEW: Failed to navigate to url " + url);
            }
        }
        else
        {
            Debug.LogError("WEBVIEW: WebView is null.");
        }
    }

    public void OnDestroy()
    {
        if (_mlWebViewScreenBehavior != null)
        {
            var webView = _mlWebViewScreenBehavior.WebView;
            if (webView != null)
            {
                webView.OnServiceConnected -= HandleOnServiceConnected;
                webView.OnServiceDisconnected -= HandleOnServiceDisconnected;
                webView.OnServiceFailed -= HandleOnServiceFailed;
            }

            _mlWebViewScreenBehavior.DestroyWebViewWindow();
        }

        if (_cancellationTokenSource != null)
        {
            _cancellationTokenSource.Cancel();
        }
    }

Another approach i tried out was to update the 1.12. example project from ML XR Provider to OpenXR XR Provider and also update to the latest MLSDK version, but unfortunately i did not manage to get the WebView running this way.

Both ways i was not able to make the WebView work with the OpenXR XR Provider and the latest MLSDK versions. Is there maybe something else you would recommend?

Thank you for your help and kind regards

[crucial-sailor]

After further investigation, i got the 1.12.1 sample project updated to OpenXR and MLSDK 2.3 and the WebView except the inputs working, but the inputs are another topic.

Nevertheless, by removing stuff part by part i found out that the MLWebViewScreenBehavior.WebView only becomes initialized after MLWebViewTabBarBehavior.CreateTab() was called.
This kind of explains the problem, but still confused me a lot since from my perspective WebViewScreenBehavior.WebView.GoTo(string url) should be able to work without creating a tab first (e.g. when i only want to use one tab all the time anyway). This wasn't clear from the documentation for me.

[etucker]

Hi @crucial-sailor,

I'm sorry that you encountered this issue and I'm glad that you were able to find a workaround. We are working on creating a better example of WebView in future versions of our SDK and we appreciate your feedback.

I will also provide a code snippet that allows you to render a URL without using tabs.

using MagicLeap.Core;
using UnityEngine;
using UnityEngine.XR.MagicLeap;

public class VerySimpleWebViewExample : MonoBehaviour
{
    [SerializeField]
    private MLWebViewScreenBehavior webViewScreenBehavior;

    private MLWebView webView;
    private string initialUrl = "https://google.com";

    private void Start()
    {
        // Check if WebView permission is granted
        if (!MLPermissions.CheckPermission(MLPermission.WebView).IsOk)
        {
            Debug.LogError("WebView permission is required.");
            return;
        }

        // Create WebView window
        if (!webViewScreenBehavior.CreateWebViewWindow())
        {
            Debug.LogError("Failed to create WebView window.");
            return;
        }

        // Get the size of the WebViewScreen and create WebView
        webViewScreenBehavior.GetWebViewSize(out uint width, out uint height);
        webView = MLWebView.Create(width, height);

        if (webView != null)
        {
            webViewScreenBehavior.WebView = webView;
            
            if (!webViewScreenBehavior.IsConnected)
            {
                webViewScreenBehavior.ServiceConnected();
            }
            
            if (!webView.GoTo(initialUrl).IsOk)
            {
                Debug.LogError("Failed to navigate to url " + initialUrl);
            }
            
            webView.OnLoadEnded += OnPageLoaded;
            webView.OnErrorLoaded += OnErrorLoaded;
            webView.OnCertificateErrorLoaded += OnCertificateErrorLoaded;
        }
    }

    private void OnPageLoaded(MLWebView webView, bool isMainFrame, int statusCode)
    {
        Debug.Log($"Page loaded with status: {statusCode}");
        webView.IgnoreCertificateError = false;
    }
    private void OnErrorLoaded(MLWebView webView, bool isMainFrame, int httpStatusCode, string errorStr, string failedUrl)
    {
        Debug.LogError($"Failed - {httpStatusCode.ToString()} - {errorStr}");
    }
    private void OnCertificateErrorLoaded(MLWebView webView, int errorCode, string url, string errorMessage, string details, bool certificateErrorIgnored)
    {
        Debug.LogError($"Cert Error - {errorCode.ToString()} - {errorMessage}");
    }

    private void OnDestroy()
    {
        // Clean up WebView when the script is destroyed
        if (webView != null)
        {
            webView.OnLoadEnded -= OnPageLoaded;
            webView.OnErrorLoaded -= OnErrorLoaded;
            webView.OnCertificateErrorLoaded -= OnCertificateErrorLoaded;
            webView.Destroy();
        }
    }
}

[crucial-sailor]

Hi @etucker ,

thank you for the code snippet, it worked great. This resolved my issue then.

[crucial-sailor]

@etucker
I have some follow up questions. I wasn't sure if i should ask them here or in a new post, so i post them here for now.

1. Is there also the possibility to run the WebView from the Unity Editor instead of on the ML? This would be very helpful during development.
2. Do you roughly know when the next WebView example will be released and whether it will directly support the OpenXR XR Provider?

Best regards

[kbabilinski]

Happy to answer these:

1. To display the WebView inside the Unity Editor, we recomend switching to a more complete 3rd party asset such as
   Vuplex
   
2. The current WebView supports OpenXR, but will remain as part of the Magic Leap SDK. We are not planning on providing WebView APIs via OpenXR Extensions.