Presented in recognition of his ground-breaking research at the interface of Artificial Intelligence (AI) and music, Gerhard Widmar’s project "Whither Music?" ("whither" is an old English word for "where") will use computers and machine learning methods to explore new methods of modeling musical abilities.

An Overall Perspective of Music
In other words, research will focus on exploring computer models for music production ("composition"), interpretation, and musical "listening" (more precisely: expectations and the predictions that listeners involuntarily form when listening to music). These computer models are intended to reproduce all of these skills at a level of quality that will support serious applications in the field of music research and music didactics. The project’s long title of "Exploring Musical Possibilities via Machine Simulation" indicates the application’s direction. These computer models will help make it possible to systematically study certain, interesting musically or music-psychologically questions as well as develop tools to support the creative development of music.

Prof. Widmer explained: "Applications like these put new demands on these types of computer models, especially in regard to quality and transparency. On one hand, they have to fulfill the musical criteria in a precise way and deliver musically plausible results. On the other hand, the predictions have to be intelligible and comprehensible. Only then can we draw useful conclusions from any experiments involving the models."

JKU Rector Meinhard Lukas added: "I would like to congratulate Prof. Widmer on his second ERC Advanced Grant! To be honored twice with a prestigious award such as this is exceptional for an academic and scientist, not to mention the research budgets for the JKU. Prof. Widmer is an outstanding researcher who has acquired groundbreaking insight into the field of excellence in artificial intelligence. In doing so, he repeatedly leaves well-laid paths within the Artificial Intelligence community but he also critically - and very reflectively – focuses on methods in Deep Learning. Taking this kind of a position is invaluable at a time of often hasty thought in all of the AI hype. On behalf of the JKU, I would like to thank him for his commitment and dedication. Researchers of his caliber are the reason for the JKU’s pioneering role in the field of Artificial Intelligence."

Markus Achleitner, State Minister for Economic Affairs, congratulated Univ. Prof. Gerhard Widmer and the JKU on the award: "Artificial intelligence is a key technology in all areas of life. By 2030, Upper Austria will be a model region for human-centered artificial intelligence. We want to focus on Artificial Intelligence that people can understand and this prestigious award brings Upper Austria one step closer to this objective, one that is firmly anchored in the #upperVISION2030 strategic program and confirms that Upper Austria is well positioned in international AI research. Professor Gerhard Widmer received this highly endowed award in recognition of his research in 2015 and for the JKU, this is the second ERC grant in less than six months."

Base-Knowledge Research in Machine Learning and Music Modeling
As an interdisciplinary, base-knowledge research project situated at the crossroads between computer science, artificial intelligence, and music, the project is based on Widmer's fundamental malaise with current mainstream in AI & Music research. “Deep Learning” dominates this area of research, meaning so-called deep neural networks are "fed" huge amounts of data, thereby learning a statistical model containing millions of parameters from the data. This is how, for example, Google Magenta's computer models learn to "compose" new music. They use existing notes in order to make note-by-note predictions. This type of research is being driven by research teams at large corporations that include Google and Amazon as they have almost unlimited computing capacities.

More Than Just Accumulating Data
Gerhard Widmer has basic doubts about the adequacy of this type of research and articulated his thoughts as part of a scientific "manifesto" back in 2017. His recent ERC project proposal contained sharp criticism. He explained, "At the moment, it’s very popular to train huge statistical optimization machines - and nothing but deep networks – over weeks at a time with enormous amounts of data (and massive amounts of electrical energy), letting them learn everything from scratch by themselves, even when talking with the broader public about AI. The resulting computer models, however, are essentially black boxes containing millions and millions of uninterpretable parameters (numbers) that don’t tell us anything about music nor can they be controlled in a transparent, comprehensible, or meaningfully way. Incidentally, this not only applies to music, but to many application areas of Deep Learning. When it comes to these kinds of models, we want to develop a fundamentally different approach by exploring just how musical knowledge can be integrated into the design and training process. The goal is to have structured and transparent models that can, in musical terms, explain and comprehend their decisions at different levels and, at the same time, demonstrate higher musical quality because they 'comprehend' music as a complex artifact at many levels of abstraction. This would be a clear turnaround in lieu of current research in AI & Music and this is exactly how those who have awarded us with the ERC grant see it as well. And what they also believe we can do."

Prof. Widmer's research team at the Institute for Computational Perception at the JKU is not recognized and known as one of the world’s leading teams for nothing.

Musical Simulation and Creative Exploration
The practical motivation as well as the project’s title is an idea to use these kinds of computer models to open up new options that can systematically explore musical possibilities and alternatives by means of "simulation". While at first it all sounds somewhat abstract, Widmer explains through (currently still imaginary) examples: "(Parameterizable) computer models that are transparent and able to be controlled in a precise way are capable of generating a certain style of musical material containing certain properties. They can also identify trends in musical data and perhaps even extrapolate them further, or predict musical probabilities which would allow us to explore potential musical developments and contexts as well as their perception and effect. How would a particular style of music have evolved under different conditions, for example, if the concept of 'euphony' and 'consonance' were different? Is it possible to make this audible? How compatible would that be in regard to our listening habits? What gives human listeners the ability to anticipate just how an unknown piece of music will continue on, or be surprised if it is different (which has a lot to do with music’s affective impact)? Can this be explained as a result of just statistical learning over a lifetime, or what abstract musical concepts would one have to assume are already present? Or interesting aspects for a more curious audience: What might the Beatles have sounded like if they had been Schubert fans? While we want to continue developing the computer as a tool to help us better understand and explore musical possibilities, we also want to creatively show all of music’s possibilities."

Widmer envisions initial research applications in music research as well as in music didactics. "As part of the project, we want to not only develop methods, we also want to collaborate with music researchers and conduct concrete studies as well as develop interactive exhibits for creative science exhibitions. This would give visitors an opportunity to experiment in a playful way and learn about the many ways that music is or could be composed." In the intermediate term, Widmer can also imagine potential applications in creative industries as well as in the world of digital music and media, for example, in music production.

Wittgenstein and Bernstein
This is Prof. Widmer’s second ERC Advanced Grant (in addition to the Austrian Wittgenstein Award). His current ERC project "Con Espressione" focuses on computer models and musical expression, which will also be applied as part of the new project.

Incidentally, the project name "Whither Music?" refers to the motto for a series of lectures by American conductor and composer Leonard Bernstein at Harvard (1973) in which he pondered the lack of alternatives (or perhaps not?) in regard to certain musical developments in a historical context.

LINKS:

European Research Council (ERC), opens an external URL in a new window

ERC Advanced Grant, opens an external URL in a new window

Homepage Gerhard Widmer, opens an external URL in a new window

Institute for Computational Perception, opens an external URL in a new window

The "Con Espressione Manifesto", opens an external URL in a new window

The Bernstein Harvard Lectures , opens an external URL in a new window

GENERAL INFORMATION:

Title:                        “WHITHER MUSIC? Exploring Musical Possibilities via Machine Simulation”

Amount:                   2.5 million euros

Duration:                      5 years (2022-2026/27)

Competitive Aspect:       2,678 applications were submitted (all European countries, across all scientific fields), of which approx. 200 were awarded with a grant, i.e. success rate for this call is approx. 7.5%.