OIT aliasing fixes

[CraigFeldspar]

• Implemented multisampling for DepthPeeling (works but has artifacts along triangle edges because we need to access multisampled depth in the shader, which cannot be done for now)
• Made depth peeling compatible with post processes to be able to use FXAA post process

[azure-pipelines]

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

[Popov72]

Great job!

What about WebGPU: does it work there too? If not and you are not able to fix it, you should create an issue for me so that I can have a look when I come back.

[EDIT] It probably won't work because in WebGPU textures are recreated when the sample count is updated and it's not possible to simply update a texture with the new value, the texture must be recreated. That means you will need to update the loop below so that the sample count is not updated multiple times for the same texture:

        for (let i = 0; i < 2; i++) {
            this._depthMrts[i].samples = value;
            this._colorMrts[i].samples = value;
        }

[Popov72]

we need to access multisampled depth in the shader, which cannot be done for now

Is it a limitation of WebGL or Babylon.js? If the latter, maybe we can have a look and see how to add the support (@sebavan)?

[sebavan]

@CraigFeldspar, any news on this PR ?

[CraigFeldspar]

@CraigFeldspar, any news on this PR ?

Hey Seb, looking at it shortly

[CraigFeldspar]

we need to access multisampled depth in the shader, which cannot be done for now

Is it a limitation of WebGL or Babylon.js? If the latter, maybe we can have a look and see how to add the support (@sebavan)?

So, this PR has 2 parts :

• Enabling postprocesses with depth peeling (so we can use FXAA)
• Enabling MSAA with depth peeling

But after the fact, I realized that MSAA has absolutely no value in our case since in WebGL2, texelFetch doesn't allow to fetch a specific sample in a multisampled fragment - see https://registry.khronos.org/OpenGL-Refpages/gl4/html/texelFetch.xhtml, with the gSampler2DMS overload. This is only OpenGL.

Therefore we will always read and write 1 depth per pixel in the depth layers, so we got no value from multisampling.

But since I already spent time on it, and it still works, even if it's unusable, I figured out it's better to leave it there until the time we can really fetch these multiple samples. That's also why I added the comment that MSAA requires RGBA, so we can be reminded if we pick back up the implementation.

I will hide the samples setter and getter on the depthPeelingRenderer, so no one mistakenly activates MSAA and wonders why it doesn't work (and makes everything slower)

[CraigFeldspar]

Great job!

What about WebGPU: does it work there too? If not and you are not able to fix it, you should create an issue for me so that I can have a look when I come back.

[EDIT] It probably won't work because in WebGPU textures are recreated when the sample count is updated and it's not possible to simply update a texture with the new value, the texture must be recreated. That means you will need to update the loop below so that the sample count is not updated multiple times for the same texture:

        for (let i = 0; i < 2; i++) {
            this._depthMrts[i].samples = value;
            this._colorMrts[i].samples = value;
        }

I think that given my above comment, we should anyway leave the multisample as it is, and pick it back if/when WebGL or WebGPU allows for multisampled texelFetch.
But maybe WebGPU already supports that ? Do you want me to create an issue to investigate ?

[CraigFeldspar]

@sebavan ready for round 2 :)

[azure-pipelines]

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

[azure-pipelines]

Snapshot stored with reference name:
refs/pull/12916/merge

Test environment:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/12916/merge/index.html

To test a playground add it to the URL, for example:

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/12916/merge/index.html#WGZLGJ#4600

Links to test babylon tools with this snapshot:

https://playground.babylonjs.com/?snapshot=refs/pull/12916/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/12916/merge
https://gui.babylonjs.com/?snapshot=refs/pull/12916/merge
https://nme.babylonjs.com/?snapshot=refs/pull/12916/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/12916/merge#BCU1XR#0

[sebavan]

I leave @Popov72 comment on it but it loooks fine to me, @Popov72 might know about the limitations regarding WebGPU

[Popov72]

It's fine to me, let's just comment out the getter and setter and we are good to go.

[CraigFeldspar]

It's fine to me, let's just comment out the getter and setter and we are good to go.

done !
@sebavan we should be good to go 🚀