Occlusion shader for built-in render pipeline - video recording issues

Unity Development
1

Hi Magic Leap!

I was checking out your latest OS and the video recording is now great, except we still have this issue:

We still use the built-in render pipeline, and I suspect occlusion shaders are rendering black because of that, since I've seen videos of other ML2 apps that don't suffer from this problem.

The thing is that we use the same shader on all the platforms we support, and it works great everywhere except in ML2, so I was hoping to get some help from your team. If we can fix this we'll be able to make amazing video recordings on Figmin XR.

This is the shader we use:


Shader "Brush/Special/Occlusion"
{
    Properties{
    }
    SubShader{
      Tags{"RenderType" = "Transparent"}
      LOD 100
      Cull off
      ZWrite On
      ZTest Less

      //occlusion pass
      Pass {
        Blend Zero One, Zero One
        CGPROGRAM
        #pragma vertex vert
        #pragma fragment frag
        #pragma multi_compile_fog
        #include "UnityCG.cginc"

        struct appdata {
          float4 vertex : POSITION;
          UNITY_VERTEX_INPUT_INSTANCE_ID
        };

        struct v2f {
          float4 vertex : SV_POSITION;
          UNITY_VERTEX_OUTPUT_STEREO
        };

        v2f vert(appdata v) {
          v2f o;
          UNITY_SETUP_INSTANCE_ID(v); //Insert
          UNITY_INITIALIZE_OUTPUT(v2f, o); //Insert
          UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o); //Insert

          o.vertex = UnityObjectToClipPos(v.vertex);

          return o;
        }

        fixed4 frag(v2f i) : SV_Target {
          return float4(0, 0, 0, 0);
        }
        ENDCG
      }

    }
}

Unity Editor version: 2022.2.19
ML2 OS version: 1.3.1
MLSDK version: unity 1.9
Host OS: windows

1 Like
2

Hi @javier we are looking into this and will get back to you shortly. Thank you for your patience.

3

Is your application using the segmented dimmer? If you are, can you confirm that the opaque visual disappears when the segmented dimmer is disabled inside you application?

4

We are actually not using the segmented dimmer, but I'm glad you asked!

The reason we aren't using it is related to this, the automatic segmented dimmer thinks that occlusion materials should activate dimming so it makes it looks terrible.

5

Hi @javier I have some bad news for you,

Firstly, It appears that Unity's built in renderer does not support catching shadows on transparent objects. The shaders that catch shadows render opaque with a transparent blendmode, but they still use the opaque render pass. This results in Magic Leap 2 capture to render the occluder incorrectly.
In summary, rendering shadows on occluder meshes is not possible when using Unity's built in render pipeline. I have reported your use case and this issue to our voice of customer team.

Secondly, we are looking into the issue with the opaque object capture right now. It looks like our Mixed Reality Capture technique and Unity's Standard render pipeline don't play nice together. If you change to URP I have a shader that would work :wink: , I will let you know as soon as I have more information.

6

Behold black magic!

It doesn't quite catch the shadows, you are right, they will cast again if they were to hit another surface lying underneath the table but that's pretty rare that it doesn't matter. This is captured on a Hololens 2 if you are curious. This is also enabled by default on the Magic Leap 2 build

But in any case, let's figure out what's the problem with mixed reality capture. We can't switch to URP, too many shaders, too many platforms, too much risk.

2 Likes
7

@javier I noticed that an older version of Figmin XR (1.37.0.0; FigminXR_ML2_v8.apk) had a built in video recording without this issue. Were there any major disadvantages of having the built-in recording functionality?

@kbabilinski very curious to hear if there's a way to get this working without the switch to URP

Thank you

8

I think the issue was always there, but the old video recordings were semi-transparent and it was harder to see. Now that the digital content is opaque it's an obvious issue.

I really hope they can fix the issue for us with the built-in render pipeline. We showcase ML2 a lot but it could be so much more if we could make use of the spatial map.