Supervision: Daniele Grattarola

Project type: Semester project (master) Master thesis

Finished

Background. Implicit neural representations (INRs) have recently emerged as a powerful way of representing signals using neural networks.

INRs are typically implemented as multi-layer perceptrons (MLPs), which are trained to map the coordinates r = [r_1, ..., r_d] of a point in some d-dimensional domain to the value of a signal at that point, f(r). For example, an INR for images would learn to map the 2D coordinates of a pixel to its RGB value. INRs have been used to learn continuous and differentiable representations for images, 3D scenes, and surfaces, and are a promising research direction for the future of AI and computer graphics.

However, it is known that simple MLPs struggle to represent high-frequency details found in most real-world signals. To overcome this spectral bias of MLPs when learning INRs, recent works have proposed to use random Fourier features or sinusoidal activation functions to facilitate the representation of the high-frequency components of the signal. A recent paper from EPFL has unified the two approaches by showing that both can be interpreted as computing a linear combination of harmonics, similar to a Fourier series expansion.

Project. This intuition of INRs allows us to ask interesting questions about them, borrowing ideas from the classical literature on signal processing.

The goal of this project is to explore the following questions (and possibly others):

1. Can we learn INRs by only training a few free parameters of the linear combination of harmonics, instead of million-parameters MLPs? If so, how can we choose the best harmonics to ensure a successful training?
2. Can we use ideas from compressed sensing to efficiently learn INRs using only a few random samples of the original signal?
3. Can we learn INRs efficiently by randomly initializing the weights of the MLP and only training the final layer? (Possibly related to point 1, but can be taken in other directions by looking at ideas from the literature on compositional pattern-producing networks).

Expected output. You will need to explore one or more of the above questions from a theoretical perspective (by reading the relevant literature), implement simple INRs in Torch/Tensorflow, and verify the theoretical insights through a series of experiments on real-world signals (images, scenes, shapes, or whatever else interests you). You will be able to reuse publicly available implementations of INRs, so you can focus more on the exploration of new ideas.

This project can be split into multiple sub-tasks suitable for semester projects and possibly a Master's thesis.

The project can lead to a publication at a top AI venue.

Profile. I am looking for Master's students with a strong background in signal processing and computer science. A basic understanding of neural networks is required (multi-layer perceptrons, differentiable programming, basic machine learning notions), as well as some practical familiarity with either Torch or Tensorflow.

Supervisor. Daniele Grattarola. I am a computer scientist with experience in geometric deep learning and computational biology. You can reach me at daniele.grattarola@epfl.ch for any questions about the project.