Ever wondered how to implement occlusion masking in UE4? I wanted this effect for my game – so I spent some time googling and ended up in Post Process material section of Unreal Engine Documentation. You might be too. There is plenty of tutorials about the use of Custom Depth and Custom Depth Stencil Value material nodes on the Internet. I personally find this youtube video quite helpful:
Tom Looman’s tutorials are pretty good too: https://www.tomlooman.com/the-many-uses-of-custom-depth-in-unreal-4/
But what I did not find and had to figure out on my own is how exactly Custom Depth and Custom Depth Stencil work. And I don’t mean how the shader itself works, but rather what kind of values these nodes return. The official documentation gives a vague description of these nodes which does not really help (I have already discussed the quality of the official documentation here: https://superyateam.com/2019/02/11/epic-is-considering-adding-an-intermediate-script-language-to-ue4-and-why-this-is-bad/)
I’ll start with Custom Depth node. The official documentation says:
This separate feature enables masking of certain objects by rendering them into another depth buffer (called custom depth buffer).
To be honest, this does not make a lot of sense to me – it does not answer what the range of values the node returns and what exactly custom depth is.
As per my understanding, custom depth returns a distance in UE4 measurement units (cm) from a player camera to an object with this feature enabled. Other objects don’t have any value in this auxiliary buffer and the node returns positive infinity for them. To prove this sort of a statement, let’s have an experiment:
We will place an object (the sphere) with Custom Depth feature enabled behind the cube on this scene:
Visually the whole scene can be split into 3 cases (from left to right on the picture above):
- The sphere which has the feature on. Custom Depth equals to Scene Depth in this case.
- The cube in front of the sphere. Scene depth of the cube is less than Custom Depth of the sphere.
- Objects with the feature disabled (by default). Custom depth returns positive infinity and Scene Depth returns an actual distance to such objects.
NOTE: There is a small issue here: Case #2 is logically the same as Case #3 – in both cases, Scene Depth is less than Custom Depth. To distinguish the cases and for the sake of the experiment, we can compare Custom Depth to some arbitrary value which is lower than positive infinity but which is greater than a distance to any object in the scene. Let it be 10 meters (1000 ue4 units):
See! We’ve got some sort of an occlusion masking. Of course, you will need to polish it before using in production, but at least it answers what Custom Depth is. Now, move on to Custom Depth Stencil.
Custom Depth Stencil
The other useful node is Custom Depth Stencil. You may use it in order to distinguish objects in a scene. By default, it returns
0 (zero) for all objects and a custom value manually set in Mesh settings.
I set Custom Depth Stencil value of 50 to the sphere on the left. And 100 – on the right. I also extended the material blueprint for the case when a mesh with Custom Depth feature is enabled – I check Custom Depth Stencil values as well too. Now, it’s just a matter of two “if” statements to figure out which color a sphere needs to be. If Custom Depth Stencil value is 50, then the color is red. If 100, then the color is white.
IMPORTANT UPDATE 29.02.2020
It turns out that CustomDepth node returns 100000000 cm rather than positive infinity when this feature is not enabled on an object. Here is a test Post Process material which, sort of, proves that:
I could not find the source code where this number comes from. So, I would appreciate it if you point me out where it is defined!