I have been putting together a proof of concept with ML2 recently and I have been using Unity and MRTK as I think it is a great tool for rapid prototyping.
I have been experiementing with meshing and was wondering how can we detect if my surface is a wall or a floor for example?
I know ML provide Plane detection from the example and api but this seem to be done with AR Foundation and an XR rig camera but since I am using MRTK, the camera and whole project setup is different and I was not able to use these feature in conjunction with MRTK. I am able to use the meshing example in conjunction with MRTK. I also noticed we can do this the MRTK way.
I was wondering:
- Can we detect surface as a wall/floor type with the meshing feature?
- If not can it be done with MRTK?
- If not, are there any other way it can be done?
- How would it work for furniture and any other type of surfaces?
Hi @jc1. I'll talk to our engineers and get you an answer asap. Thanks for your post.
@jc1 The Magic Leap 2 does not support plane finding via MRTK. Please use the AR Foundation APIs to detect the surface types of each plane.
You can add the XR Origin , AR Plane Manager , AR Session and AR Input Manager components to the MixedRealityPlayspace .
Make sure the Tracking Origin Mode is set to Device and the Y Offset is set to 0. and set a Plane Prefab inside the AR Plane Manager. Once this is complete you should see plane populating inside your scene.
One thing to note is that we noticed a bug in one of Unity's packages that may not report the correct planes correctly when using the AR Raycast Manager. We are working quickly to fix this, as a temporary workaround use the Plane Manager's Raycast directly. Here is an example of casting a ray from the controller and getting the hit type of collided plane. Note: the following sample was not analyzed for performance.
public class PlaneRaycastExample : MonoBehaviour
private MagicLeapInputs mlInputs;
private MagicLeapInputs.ControllerActions _controllerActions;
private ARPlaneManager arPlaneManager;
arPlaneManager = FindObjectOfType<ARPlaneManager>();
if (arPlaneManager == null)
Debug.LogError("Missing AR Plane Manager. Please make sure it is present in the scene hierarchy.");
enabled = false;
mlInputs = new MagicLeapInputs();
_controllerActions = new MagicLeapInputs.ControllerActions(mlInputs);
_controllerActions.Bumper.performed += HandleOnBumper;
private void HandleOnBumper(InputAction.CallbackContext obj)
bool bumperDown = obj.ReadValueAsButton();
Debug.Log("The Bumper is pressed down " + bumperDown);
var raycastRay = new Ray(_controllerActions.Position.ReadValue<Vector3>(),
_controllerActions.Rotation.ReadValue<Quaternion>() * Vector3.forward);
var hits = arPlaneManager.Raycast(raycastRay, TrackableType.PlaneWithinPolygon, Allocator.Temp);
var orderedHits = hits.OrderBy(_ => _.distance).ToList();
for (int i = 0; i < 1; i++)
Debug.Log("Raycast Hit " + orderedHits[i].hitType);