Feature attribution is often presented as the process of selecting a small subset of features most responsible for a prediction. Task-dependent by nature, precise definitions of responsibility encountered in the literature are however not always consistent. This lack of clarity stems from the fact that we usually do not have access to any notion of ground-truth attribution and from more general debates on what good interpretations are. We propose to formalise feature selection/attribution based on the concept of relaxed functional dependence. In particular, we extend our notions to the instance-wise setting and derive necessary properties for candidate selection solutions, while leaving room for task-dependence. By computing ground-truth attributions on synthetic datasets, we evaluate many state-of-the-art attribution methods and show that, even when optimised, some fail to verify the proposed properties and provide wrong solutions.
"In this talk, first we look at how different inductive biases can be incorporated into machine learning systems, and what are the consequences of such inductive biases. We then review some of the widely-used inductive biases in deep learning models, and try to understand how they help models improve their prediction performance. More specifically, we look at popular data augmentation techniques, and their inductive biases. Further, we look at how the decision boundaries of deep neural networks can cause deep neural networks to be more vulnerable to adversarial attacks, and how this can be systematically and objectively measured. And in the end, we look at how data augmentation techniques affect decision boundary of deep neural networks, from two different aspects: 1) incorporating an inductive bias, and 2) incorporating stochasticity into training data. Using tools from statistics, we disentangle these factors, and try to analyze their effects independently, using our proposed analysis framework."