1. Could you please introduce yourself?

I grew up in Switzerland in a both musical and scientific environment. During my Computer Science studies at the University of Bern, I became increasingly interested in theoretical aspects of music and music history and attended musicology classes in my free time. After my studies, I gained some experience in the industry working as a software engineer, while also frequently performing as a musician. When I was offered a PhD position that allowed me to combine my two primary passions, I decided to go back to academia. I spent the following years at the University of Minnesota to pursue a PhD in Music Theory. After my studies, I joined the Centre for Digital Music and the FAST project as a postdoctoral researcher at Queen Mary in London.

2. What is your role/work within the project?

Within FAST I’m mainly concerned with the consumption side of things, particularly with enabling new kinds of experiences during music discovery and music listening. I’m working on new ways of automatically analyzing music and representing it in a semantically annotated and flexible way so that it can be compared, browsed, and played back nonlinearly, i.e. dynamically reorganized to adapt to the listener’s context or respond to user interaction. I’m involved in various research and artistic collaborations within the project and beyond, where these technologies are being tested and used creatively.

3. What, in your opinion, makes this research project different to other research projects in the same discipline?

The FAST project is highly interdisciplinary and unites three research groups with very different but complementary specializations, which nurtures our equally interdisciplinary goals by bringing together signal processing, studio technology, the semantic web, artificial intelligence, workflow analysis, ethnography, and design. We are possibly the only project considering the entire value chain of the music industry, from composition, via recording, production, and distribution, to consumption. One of our main goals, using linked data to enable a musical entity to travel through the entire music value chain while accumulating more and more information that will be useful at various stages, can only be achieved through the collaborative work of people with such varied and creative backgrounds, as well as with our industrial partners.

4. What are the research questions you find most inspiring within your area of study / field of work?

With my background in music theory, I’m thrilled by the possibilities of automatically analyzing large collections of musical material and, for instance, finding trends and long-term developments in a particular artist’s or group’s work, similarities and connections with other artists, or differences in how a work is performed on different days and/or in different environments.

From the point of view of education, I’m interested in enabling music discovery and teaching through music listening itself, e.g. by highlighting particular musical aspects directly within the audio, instead of using distracting and abstract visualizations detached from the musical content. I’m also interested in educational and creative uses of sonification in order to, for example, present results of search queries in an auditory way.

As a performer, I’m mainly interested in coming up with new ways of interacting with musical material using physical and gestural interfaces, while trying to match the versatility of traditional graphical and textual computer interfaces. From this perspective, the use of machine learning in realtime is particularly interesting, for I’ve found that such systems often behave in unpredictable, seemingly creative ways.

5. What, in your opinion, is the value of the connections established between the project research and the industry?

It’s a great opportunity to not only discuss the interests and requirements of our industry partners first-hand, but also improve our outreach by implementing prototypes in collaboration with our partners, which could then be further developed for specific applications in the industry. Sometimes new developments in the industry can be difficult to oversee and they often remain concealed until they are launched publicly. It’s therefore very valuable to have our ideas scrutinized at an early stage and from an economic point of view by our partners.

6. What can academic research in music bring to society?

Musical practice has always been surprisingly quick to embrace new technological developments, both in experimental circles and the mainstream. Many technological ideas emerge and are first elaborated in academic contexts, to then be applied in practice and distributed in an economic context, e.g. by startup companies. These, in turn, make it possible for new musical tools and devices to become accessible and affordable by a larger audience of musicians and consumers, who almost always use the technology in creative ways that were initially not envisioned. It’s very exciting to see this process.

Besides having a potential technological influence on the music of the future, perhaps more importantly, academic research can further the understanding of our incredibly rich musical heritage. Research not only contributes to our common knowledge about music, but it can help to organize existing musical knowledge and public music collections and make them accessible and understandable by a large community of musicians, students, researchers, and music lovers. These aspects are of course equally relevant to the music industry who may find new ways of recommending and distributing music, as well as creating new experiences.

7. What are you working on now?

I’m currently working on pattern recognition algorithms and music similarity models that allow us to automatically recognize and analyze various kinds of musical form and structure. I’m also working on finding appropriate ways to represent such musical structures as semantically annotated hierarchical and graph-based structures so that they can be queried, compared, and navigated. These algorithms and models are being applied in several projects within FAST.

For example, we’re looking at the Grateful Dead collection in the Live Music Archive, an incredibly valuable collection of live recordings from the early days of the group in the sixties to its disbandment in the mid-nineties. We are using the algorithms on various subsets of the collection. For example, we align different tape recordings of a single concert, spatialize them, and create an immersive experience in which the listener can move around. We also compare different performances of the same song across decades and allow listeners to move through time while listening to a song.

We’re also working on reorganizing and transforming recordings of various nature. We first decompose the recordings into single sound events using source separation algorithms and then put these sounds back together dynamically. This enables us to do things such as spatialize the contents of a mono recording around the listener to highlight some musical aspects or distort a recording in time to zoom in on certain performance aspects.

As a foundation of all this, I’m building a prototype of a mobile and web-based player, the Semantic Player, which uses these algorithms and representations to find out how any provided musical material can be played back in a malleable and dynamic way. The player can receive any input from mobile device sensors and data from the web, which can then be used to modify the music. We’re currently experimenting with all kinds of experiences where, for instance, the music adapts to contextual conditions such as the weather at the listener’s current location.

8. Which is the area of your practice you enjoy the most?

I really enjoy thinking about specific problems and trying to come up with solutions that can then be generalized to other problems or situations. I also enjoy discussing problems with other researchers and collaborating to try to bring together different parts of research in a common application. In FAST, we are experiencing this all the time when working on our joint demonstrators. Most importantly, I’m almost always driven by an idea of something I would like to use or listen to myself. Nothing is more satisfying to me than implementing a new methodology and testing it experimentally, playing around with it, or even performing with it, and then improving and optimizing it iteratively.

9. What is it that inspires you?

A lot of my inspiration comes from other disciplines and thinking about how they connect. I love creating analogies between situations in another field of application and the one I’m currently concerned with. There is always someone that has tried similar things before, however their work can easily be overlooked if it was conducted in a different field, so bridging the gap between research in different disciplines is important. Most importantly, I get a lot of inspiration from musical practice, nature, and the arts in general.