Foveated Path Tracing & Denoising on HMD with Eye Tracking
We engineer the Nvidia OptiX path tracing algorithms into the FOVE framework, and achieve higher frame rate by reducing the number of photons in the peripheral regions. Finally, using spatio-temporal wavelet and variance filters, our pipeline significantly eliminates the noise of the incomplete path tracing, and improves the rendering quality with foveation. From the experimental results, we demonstrate that our foveated rendering pipeline achieves approximate 39 speedup than the original while producing visually similar results.
Click the image for more!
Denoising Videos with Convolutional Autoencoders
One way machine learning can improve ray tracing efficiency is to use reinforcement learning to progressively learn where light comes from in order to guide the light ray sampling strategy. This method results in faster convergence to a noise-less image. Another approach is to use a neural network architecture called a convolutional autoencoder to denoise images rendered with a low sample count per pixel.
The latter post-processing approach is the focus of this project.
Click the image for more!
My Auto Panorama!
The aim of this project is to implement an end to-end pipeline to do image panorama stitching of unordered images.
Face Swap!
The aim of this project is to implement an end to-end pipeline to swap faces of videos and images.
3D Reconstruction and Segmentation
In this project, efficient and robust algorithms for 3D object reconstruction and segmentation are implemented.
