Klangsalat

Auditory Navigation System

The Concept

Klangsalat is a geolocation based Android app, that enables its users to discover the city of St. Pölten in a new way: with their ears! Several points of interest are marked with contemporary or historical sounds that the users hear as soon as they move in range of the specific point. Klangsalat uses 3D Audio to map the sounds to their associated places and to make the users hear the sounds from the correct direction based on their location and viewing angle.

Here’s a small video to showcase Klangsalats functionality. Enjoy! The video was created by Tamás Künsztler, kudos!

Technologies & Techniques

Another project with a first time used technology for me: Android. My Java skills were quite basic beforehand, though with a little training I was able to develop the needed features for this working prototype. The audio processing is done in Pure Data, an open source visual programming language for multimedia. It helped us to integrate the HRTFs (head related transfer functions) needed for the 3D audio processing. With the library libpd we were able to include the Pd patch in the Java source code and thus unite Android and Pd.

A lot of newly used features and programming languages also have their downsides: Klangsalat is what it is – a prototype. It uses a lot of disk space due to the lack of support for compressed audio files in libpd and it is very processing intensive which makes it not really usable on weaker smartphones.

The Story

This was a semester project during the second semester of my study in Digital Media Technologies. The aim was to create something related to “Sonic Interaction”. After going through a lot of different ideas, we decided to do something very up to date: use 3D audio in a mobile app and combine it with geolocation based data. As I mentioned we weren’t at all experts in app development, nor did we do a lot with binaural audio at that time. But we were able to create that working prototype and we proved that using audio as a (kind of) navigational method is possible. The scientific approach led to the creation of a small paper that we submitted at the Forum Medientechnik 2016. The jury allowed us to present it at the event as well and the audience seemed to be impressed by our presentation and the project.

As I mentioned we were able to create a prototype, but we also proved that 3D audio is not too far from becoming relevant in mobile development and could be a great supplement to games or other location based apps already on the market, like Pokémon Go & Co. If you are interested in how it all works on the technical side, be free to visit my GitHub profile and check it out.

 

Leave a Comment

Error: Please check your entries!
Back To Top