Ilia Sucholutsky

Summary

I’m fascinated by deep learning and its ability to reach superhuman performance on so many different tasks. I want to better understand how neural networks achieve such impressive results… and why sometimes they don’t. To do that, I’m exploring what kind of information or knowledge is contained in the datasets we train our models on and how much of this knowledge is actually needed for our models.

In the beginning, I used deep learning to restore lost data from cars in order to improve anomaly detection algorithms and make cars safer. The ability to restore lost data suggests that knowledge is duplicated across a dataset.

Then, I worked on improving dataset distillation, the process of learning tiny synthetic datasets that contain all the knowledge of much larger datasets. If knowledge is duplicated across a dataset then it should be possible to represent that knowledge using fewer samples.

Now, I work on “less than one”-shot learning, an extreme form of few-shot learning where the goal is for models to learn N new classes using M < N training samples. If models can generalize from a small number of synthetic samples, can they also generalize from a small number of real samples?

Algorithms & Analysis of Algorithms Artificial Intelligence Data Mining & Machine Learning Real-Time & Embedded Systems Statistics

Past or current institution affiliations