JKU LIT at the Ars Electronica Festival: Music Tower Blocks

A city of music just the way you want it. Music Tower Blocks (EmoMTB) is a LIT project that gives visitors the opportunity to build a “city” with music and explore large collections of music in an audio-visual way. The city’s buildings consist of similar music and the different districts are composed of similar music styles. We spoke about EmoMTB with Prof. Markus Schedl, Institute for Computational Perception at the JKU and project manager.

Markus Schedl

Can you briefly describe your project?
Markus Schedl: EmoMTB is a music exploration interface capable of displaying a collection of half a million music tracks on a large screen in the form of a city. The visitor can explore the city and use a smartphone like a navigator – similar to a gamepad - to play the music from his or her own Spotify playlist. At the same time, the incorporated AI can analyze the visitor’s mood/emotional state to create corresponding music recommendations which can subsequently be incorporated into the visualization, making it possible to find the songs and music that reflects the visitor’s current mood.

Where did the idea or inspiration come from?
Markus Schedl: We have been working on intelligent user interface systems to support music visualization as well as to explore music collections for years now. The original idea to automatically group music pieces by using machine learning methods and presenting them in a visually compelling way all began with the "Islands of Music" and our adaptation, "nepTune". Metaphorically, we originally selected an island landscape with the idea of grouping similar music on an ‘island’. As we were creating EmoMTB, the ‘island’ concept eventually gave way to buildings and musical songs stacked on top of each other, forming individual floors of a building. Compared to similar, previous interfaces, EmoMTB is can also handle fairly large collections of music. And, of course, including a mood aspect is something new and different as well.

What are the project participants’ background?
Markus Schedl: The EmoMTB team’s background is very diverse, featuring experts in AI, media art, game design, and musicology. Our team also includes experts in the fields of machine learning and affective computing as well as conventional computer scientists and software developers.

What does "A New Digital Deal" mean when it comes to your project; or, what would you like to see as part of a "New Digital Deal"?
Markus Schedl: The AI technologies used today, for example, are based on web search engines and recommendation systems and tend to be "biased" in many different ways. On one hand, modern Deep Learning models are usually trained using biased data; on the other hand, the algorithms themselves often amplify already existing biases. These kinds of biases often result in treating entire user groups unequally, particularly in terms of gender, age, etc.
As part of a "New Digital Deal", I would like to see these system providers – especially the ones we use on a daily basis - commit to clearly communicating biases that result in unequal treatment and minimize these biases by making use of new technological options. We are actively researching these kinds of fair technologies and hope they will be widely implemented soon.

What can visitors look forward to most when they come to see your project?
Markus Schedl: A cool, new way to explore their playlist and maybe discover completely new music, especially things off of the beaten track.

What have been some of the greatest challenges when implementing your project?
Markus Schedl: The biggest challenge was clearly mastering the discrepancy between everything we wanted to make happen and what was actually feasible using resources and technologies at hand.

Learn more about JKU Projects at the 2021 Ars Electronica Festival

Interview Prof. Markus Schedl

We would like to point out that when playing the video, data may be transmitted to external parties. Learn more by reading our data privacy policy