There are a number of Approaches in Intelligent Music Production – as discussed in my recent journal paper (here) – but the key aspects to consider are
- How can I interact with the intelligent musical tool?
- How does it understand what I am doing?
- How can I get it to change the sounds?
Until now, most intelligent music production has been all about automating the parameters of a single audio effect – eg. the cut-off frequency of an EQ filter – but – fundamentally, each individual aspect of the audio processing will clearly impact every other part of the mix – Why should we be restricted to traditional audio signal processing chains? Why should we rely on an EQ, which was first designed 70 years ago? Surely, with all the AI and intelligent systems in the world, computers can find much better – or even… just more interesting – ways of interacting with and shaping audio.
Having recently attended DAFx, for my 4th year, I was presenting my paper on Objective Evaluation of Synthesised Environmental Sounds.
The basic premis of the paper, is that we can computational measure how similar two sounds are using an objective metric. This objective metric can be evaluated using an iterative resynthesis approach. And a given similarity score can be evaluated through comparison to human perception.
I hope this made sense, but if not please get in touch and I would be happy to explain further. The paper will be available on the DAFx Website shortly.
At the upcoming International Conference on Digital Audio Effects, I will be presenting my recent work on creating a sound effects taxonomy using unsupervised learning. A link to the paper can be found here.
A taxonomy of sound effects is useful for a range of reasons. Sound designers often spend considerable time searching for sound effects. Classically, sound effects are arranged based on some key word tagging, and based on what caused the sound to be created – such as bacon cooking would have the name “BaconCook”, the tags “Bacon Cook, Sizzle, Open Pan, Food” and be placed in the category “cooking”. However, most sound designers know that the sound of frying bacon can sound very similar to the sound of rain (See this TED talk for more info), but rain is in an entirely different folder, in a different section of the SFx Library.
Our approach, is to analyse the raw content of the audio files in the sound effects library, and allow a computer to determine which sounds are similar, based on the actual sonic content of the sound sample. As such, the sounds of rain and frying bacon will be placed much closer together, allowing a sound designer to quickly and easily find related sounds that relate to each other.
A full run down of the work is present on the Intelligent Audio Engineering Blog
I have been accepted to publish my MSc project on Dereverberation applied to Microphone Bleed Reduction.
I implemented existing research in reverb removal and combined with with a method for microphone interference reduction. In any multiple source environment there will interference from opposing microphones as pictured below.
Research at Queen Mary University allows this interference to be reduced in real time processing and my project was to improve this with the addition of removing natural acoustic reverberation in real time, to assist with the microphone bleed reduction.
This work will be published at the AES conference on DREAMS (Dereverberation and Reverberation of Audio Music and Speech).
David Moffat and Joshua D. Reiss. “Dereverberation and its application to the blind source separation problem”. In Proc. Audio Engineering Society Conference: 60th International Conference: DREAMS (Dereverberation and Reverberation of Audio, Music, and Speech). Audio Engineering Society, 2016. to appear.
The 8th September 2015 sees the Audio Engineering Society UK Midlands Section presenting a workshop on Intelligent Music Production at Birmingham City University.
As ever, C4DM have a strong presence at this workshop, as two of the six presented talks are by current C4DM members. Ryan Stables, the event organiser, and others at the Digital Media Technology (DMT) Lab in Birmingham City University are currently collaborating with C4DM on the Semantic Audio Feature Extraction (SAFE) project. More information on this project can be found here
Josh Reiss will present a summary of the current state of the art in Intelligent Music Production, highlighting current research directions and the implications of this technology. Brecht De Man will present some of his PhD results in perceptual evaluation of music production as he attempts to understand how mix engineers carry out their work. Further to this, Alex Wilson was a previous C4DM visiting student for six months, and will be presenting his recently publishing work from Sound and Music Computing Conference, in navigating the mix space.
More information on the workshop, including abstracts and registration, can be found here http://www.aes-uk.org/forthcoming-meetings/aes-midlands-workshop-on-intelligent-music-production/.
Today, 28th August 2015, C4DM presented a one day workshop entitled Listening In The Wild, organised by Dan Stowell, Bob Sturm and Emmanouil Benetos.
The morning session presented a range of research including sound event detection using NMF and DTW techniques, understanding detectability variations of species and habitats, animal vocalisation synthesis through probabilistic models.
The post lunch session saw discussion on vocal modelling and analysis working towards understanding how animals produce their given associated sounds. Following this there was further discussion on NMF followed by work on using bird songs as part of a musical composition.
The poster session included work on auditory scene analysis, bird population vocalisation variations, CHiME: a sound source recognition dataset, technology assisted animal population size measures, bird identification through the use of identity vectors, and DTW for bird song dissimilarity.
Further information on the presenters and posters is avaliable here
There are a range of interesting and exciting events that are upcoming in the field audio technology, including:
Listening in the Wild – A machine listening workshop hosted at Queen Mary University on the 25th of June. This will discuss how animals and machines can listen to complex soundscapes. More information here: http://www.eecs.qmul.ac.uk/events/view/listening-in-the-wild-animal-and-machine-hearing-in-multisource-environment
Intelligent Music Production – A workshop presented at Birmingham City University on the 8th September on the current state of the art in audio production technology, perception and future implications. Details are here: http://www.aes-uk.org/forthcoming-meetings/aes-midlands-workshop-on-intelligent-music-production/
Both of these events are free to attend, and promise to look very exciting indeed.