Go to JKU Homepage
Linz Institute of Technology
What's that?

Institutes, schools, other departments, and programs create their own web content and menus.

To help you better navigate the site, see here where you are at the moment.

Modern Hopfield Networks and Attention for Immune Repertoire Classification

Publication by Institute for Machine Learning and LIT AI Lab in cooperation with Department of Immunology and Department of Informatic, University of Oslo, Norway and Institute of Advanced Research in Artificial Intelligence (IARAI): "Immune repertoire classification with attention-based deep massive multiple instance learning" by Michael Widrich, Bernhard Schäfl, Milena Pavlović, Hubert Ramsauer, Lukas Gruber, Markus Holzleitner, Johannes Brandstetter, Geir Kjetil Sandve, Victor Greiff, Sepp Hochreiter, Günter Klambauer. Accepted as publication and spotlight talk at NeurIPS 2020.


Schematic representation of the DeepRC approach

In immune repertoire classification, a vast number of immune receptors are used to predict the immune status of an individual. This constitutes a MIL problem with an unprecedentedly massive number of instances, two orders of magnitude larger than currently considered problems, and with an extremely low witness rate. Accurate and interpretable machine learning methods solving this problem could pave the way towards new vaccines and therapies, which is currently a very relevant research topic intensified by the COVID-19 crisis. In this work, we present our novel method DeepRC that integrates transformer-like attention, or equivalently modern Hopfield networks, into deep learning architectures for massive MIL such as immune repertoire classification. We demonstrate that DeepRC outperforms all other methods with respect to predictive performance on large-scale experiments including simulated and real-world virus infection data and enables the extraction of sequence motifs that are connected to a given disease class. Source code and datasets: https://github.com/ml-jku/DeepRC, opens an external URL in a new window