Continuing on yesterday’s work.

The “specks” look like dithering
The “specks” look like dithering

I added a hole and speck fix for the depth-based occlusion. The depth-based occluder had the same issues the color overlay on the infrared camera had since those issues are the limitations of an infrared sensor; I just never noticed the issues before because the depth occlusion of particles is mostly invisible. Even though the attempt to fix the grey holes in the colored infrared feed failed horribly, I was able to get a similar patch working with the depth occlusion because both of those feeds (infrared and depth) share an aspect ratio (so there was no alignment offset). To patch a hole efficiently, the shader reads the depth of the nearest edge to the first empty pixel it reads and fills the whole hole at that depth. Claude even moved the hole fill to a background thread to save a few fractions of a millisecond. Beast.


Moving on to the rendered recording…I realized it was kind of dumb of me to choose a video filmed in broad daylight as a background for my particles overlay. To address this, I added a background-dimming shader while keeping the subject fully-lit by superimposing the subject mask on a higher layer. Still, I had to make my particles more visible. Aside from increasing their size, I noticed that post processing wasn’t working like it was supposed to on the particles. Claude found the issue, which was that “the project’s URP Alpha Processing applied color grading in proportion to each pixel’s alpha while the additive particles wrote almost no alpha, so saturation, exposure, white balance and the like barely reached them, and never reached the bloom glow around them. To fix this, it created a new renderer feature that only runs on the Particles Camera. Before post-processing, it saves the particle layer and makes it fully opaque, so the grade applies everywhere. Empty pixels are black and stay black. After post-processing, it restores the saved alpha, so the layer still adds light over the feed exactly as before.”

After some tweaking to the post-processing and particle settings, I had Claude capture a recording using the Unity recorder. I freaking love the results and can’t wait to post them on IG.


Tags: rendering depth-camera shaders particles footage