Masterthesis (f/m/x) - Neural Horizon Mapping

The Research Project

We develop CosmoScout VR, an open source software for rendering realistic images of large scale planetary surfaces in real-time. Applications include mission planning, generation of training data for pose estimation and immersive analysis of georeferenced datasets. One key criteria for realism is the presence of accurate shadows, however these are still challenging to produce for arbitrary scenes in real-time. We are looking for efficient ways of encoding and storing information required for computing large scale shadows for arbitrary lighting and viewing conditions.

The Research Questions

Can neural representations be used to efficiently store and access occluder information of planetary terrains for computing shadows at runtime?

Lately, machine learning approaches have been used in various parts of the rendering pipeline to achieve high-quality visuals at a significantly lower memory footprint compared to non-ML pipelines. A key method is Neural texture compression: Here, a neural network is trained to reconstruct surface texture data from a compact latent representation. The core of this thesis is verifying the viability of applying this approach to so-called horizon maps, i.e. auxiliary textures used in computing self-shadows of planetary terrains.

Your Tasks

• Design and train a neural network for compressing and decompressing horizon maps
• Use the neural network to render shadows on a planetary surface in real-time
• Perform quantitative evaluations of performance and quality