Temporally Stable Metropolis Light Transport Denoising using Recurrent Transformer Blocks

Chuhao Chen1, Yuze He2, Tzu-Mao Li1,
1University of California San Diego, 2Tsinghua University

ACM Transactions on Graphics (SIGGRAPH 2024)

Paper Video Code(coming soon) Dataset(coming soon)

The illustrated example is one middle frame from a 60-frame MLT animation of the scene Monkey. The left-bottom crop is a detailed region frame and the right-bottom crop is an average over the 7 consecutive frames in this detailed region. In this example, the state-of-the-art sequence Monte Carlo denoisers (b) suffers from large bias, excessive blur and square artifacts from dilated kernels, while our denoiser (c) produces high-quality results and keeps the temporal stability. All the metrics are calculated on the whole animation.

Abstract

Metropolis Light Transport (MLT) is a global illumination algorithm that is well-known for rendering challenging scenes with intricate light paths. However, MLT methods tend to produce unpredictable correlation artifacts in images, which can introduce visual inconsistencies for animation rendering. This drawback also makes it challenging to denoise MLT renderings while maintaining temporal stability. We tackle this issue with modern learning-based methods and build a sequence denoiser combining the recurrent connections with the cutting-edge vision transformer architecture. We demonstrate that our sophisticated denoiser can consistently improve the quality and temporal stability of MLT renderings with difficult light paths. Our method is efficient and scalable for complex scene renderings that require high sample counts.

Video

BibTeX

@article{Chen:2024:MLTD,
  author = {Chen, Chuhao and He, Yuze and Li, Tzu-Mao},
  title = {Temporally Stable Metropolis Light Transport Denoising using Recurrent Transformer Blocks},
  journal = {ACM Transactions on Graphics (Proceedings of SIGGRAPH)},
  year = {2024},
  volumn = {43},
  number = {4},
  articleno = {123}
}