Deep Representation of a Normal Map for Screen-Space Fluid Rendering
Source abstract
Abstract
We propose a novel method for addressing the problem of efficiently generating a highly refined normal map for screen-space fluid rendering. Because the process of filtering the normal map is crucially important to ensure the quality of the final screen-space fluid rendering, we employ a conditional generative adversarial network (cGAN) as a filter that learns a deep normal map representation, thereby refining the low-quality normal map. In particular, we have designed a novel loss function dedicated to refining the normal map information, and we use a specific set of auxiliary features to train the cGAN generator to learn features that are more robust with respect to edge details. Additionally, we constructed a dataset of six different typical scenes to enable effective demonstrations of multitype fluid simulation. Experiments indicated that our generator was able to infer clearer and more detailed features for this dataset than a basic screen-space fluid rendering method. Moreover, in some cases, the results generated by our method were even smoother than those generated by the conventional surface reconstruction method. Our method improves the fluid rendering results via the high-quality normal map while preserving the advantages of the screen-space fluid rendering methods and the traditional surface reconstruction methods, including that of the computation time being independent of the number of simulation particles and the spatial resolution being related only to image resolution.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Computer Graphics and Visualization Techniques · Advanced Vision and Imaging · Generative Adversarial Networks and Image Synthesis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science and Engineering, Korea University, Seoul 02841, Korea
Interdisciplinary Program in Visual Information Processing, Korea University, Seoul 02841, Korea
Funding
National Research Foundation of Korea · Ministry of Trade, Industry and Energy
References (37)
Show source-provided references
1. Van der Laan, W.J., Green, S., and Sainz, M. (March, January 27). Screen space fluid rendering with curvature flow. Proceedings of the 2009 Symposium on Interactive 3D Graphics and Games, Boston, MA, USA.10.1145/1507149.1507164 ↗
2. A Narrow-Range Filter for Screen-Space Fluid Rendering10.1145/3203201 ↗
3. Radford, A., Metz, L., and Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv.
4. Isola, P., Zhu, J.Y., Zhou, T., and Efros, A.A. (2017, January 21–26). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.632 ↗
5. Mirza, M., and Osindero, S. (2014). Conditional generative adversarial nets. arXiv.
6. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial nets. Advances in Neural Information Processing Systems, MIT Press.
7. Marching cubes: A high resolution 3D surface construction algorithm10.1145/37402.37422 ↗
8. Reconstructing surfaces of particle-based fluids using anisotropic kernels
9. Zwicker, M., Pfister, H., Van Baar, J., and Gross, M. (2001). Surface splatting. Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, ACM.10.1145/383259.383300 ↗
10. Aurich, V., and Weule, J. (1995). Non-linear Gaussian filters performing edge preserving diffusion. Mustererkennung 1995, Springer.10.1007/978-3-642-79980-8_63 ↗
11. Müller, M., Schirm, S., and Duthaler, S. (2007, January 2–4). Screen space meshes. Proceedings of the 2007 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, San Diego, CA, USA.
12. Xiao, X., Zhang, S., and Yang, X. (2017, January 25–27). Real-time high-quality surface rendering for large scale particle-based fluids. Proceedings of the 21st ACM Siggraph Symposium on Interactive 3D Graphics and Games, San Francisco, CA, USA.10.1145/3023368.3023377 ↗
13. Burkus, V., Kárpáti, A., and Szécsi, L. (2021). Particle-Based Fluid Surface Rendering with Neural Networks, Union.10.24132/CSRN.2021.3002.26 ↗
14. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3–6). Imagenet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA.
15. Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv.10.3115/v1/D14-1179 ↗
16. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
17. Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.10.1007/978-3-319-24574-4_28 ↗
18. Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder
19. Deep shading: Convolutional neural networks for screen space shading10.1111/cgf.13225 ↗
20. Sanchez-Gonzalez, A., Godwin, J., Pfaff, T., Ying, R., Leskovec, J., and Battaglia, P. (2020, January 12–18). Learning to simulate complex physics with graph networks. Proceedings of the International Conference on Machine Learning, Virtual Event.
21. Graph neural networks in particle physics10.1088/2632-2153/abbf9a ↗
22. Pfaff, T., Fortunato, M., Sanchez-Gonzalez, A., and Battaglia, P.W. (2020). Learning mesh-based simulation with graph networks. arXiv.
23. Machine learning–accelerated computational fluid dynamics10.1073/pnas.2101784118 ↗
24. Dosovitskiy, A., and Brox, T. (2016, January 4–9). Generating images with perceptual similarity metrics based on deep networks. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain.
25. Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22–29). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.244 ↗
26. Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., and Wang, Z. (2017, January 21–26). Photo-realistic single image super-resolution using a generative adversarial network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.19 ↗
27. tempogan: A temporally coherent, volumetric gan for super-resolution fluid flow
28. Wang, T.C., Liu, M.Y., Zhu, J.Y., Liu, G., Tao, A., Kautz, J., and Catanzaro, B. (2018, January 3–8). Video-to-Video Synthesis. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Montreal, QC, Canada.
29. Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., and Efros, A.A. (2016, January 27–30). Context encoders: Feature learning by inpainting. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.278 ↗
30. Loss functions for image restoration with neural networks10.1109/TCI.2016.2644865 ↗
31. Müller, M., Charypar, D., and Gross, M. (2003, January 26–27). Particle-based fluid simulation for interactive applications. Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, San Diego, CA, USA.
32. Position based fluids
33. Rectifier nonlinearities improve neural network acoustic models
34. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
35. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
36. Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., and Isard, M. (2016, January 2–4). Tensorflow: A system for large-scale machine learning. Proceedings of the 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16), Savannah, GA, USA.
37. Thomas, M.M., and Forbes, A.G. (2017). Deep Illumination: Approximating Dynamic Global Illumination with Generative Adversarial Network. arXiv.