Tune In and Listen to Your Surroundings


Many mobile users listen to music in order to free their minds in the constrained space and time. However, the isolated auditory bubbles make them become further disconnected from the world. How can we use sound as the medium to connect the mobile user to the environment? How can the linking experience be smooth, easy, yet personal? Can the system still be effective on an extremely dense audio map? How can we enhance the simultaneous listening experience based on context?
Loco-Radio is an AR auditory environment for car drivers, bikers, and pedestrians. The mobile user is able to tune in and listen to the surroundings. The system augments nearby places with localized audio streams and immerses the user within a soundscape as he/she moves. It works indoor and outdoor, and is designed at street and building scale. The research explores context-aware strategies of using spatial audio in high-density audio environments.

For Outdoor
For Indoor


Loco-Radio Outdoor creates an AR auditory experience that enhances the mobile users' awareness of the nearby restaurants. The environment immerses them within a soundscape. As users move on the street, they encounter a series of songs and the perception enhances their awareness of the numbers, styles, and locations of restaurants. The experience helps them construct a mental food map.

The Audio Map

A compact audio map is created by associating genre-matching songs to restaurants. The place information is collected by using Google Place and Yelp API. The database covers 392 restaurants in Cambridge/Somerville MA, and each is attached with a genre-matching song. The below figure shows the restaurants around MIT. Most of them locate near Central Square. In order to overcome the uneven distribution of sound streams, auditory scaling is necessary.

Auditory Spatial Scaling

Auditory scale defines the relations between sound and space; it describes how sounds are heard by mobile users in augmented space. Scaling alters the relations and can be used to transform the auditory experience. Various techniques were introduced: automatic and manual zooming, asymmetric scaling, and stereoized crossfading. They allow designers to create effective UI for AR audio environments with a large amount of simultaneous streams.

The above figure visualizes the automatic zooming process. Each circle represents the effective audible range at the location. The scale is dynamically adjusted so that the user is not overwhelmed by a large number of simultaneous streams.


The AR audio is more fragmented when moving at a faster speed. Therefore, the hearing range needs to be increased with a fast moving speed. In addition, the motion of car and bicycle are more predictable. Therefore, we can play farther along the direction of the user's motion.

The Study

Loco-Radio Outdoor was evaluated by 10 subjects (drivers, bikers, pedestrians) in a think aloud study. The study showed that since the user tended to confirm the location of sound visually, it created double impression and allowed the user to remember the place well. Biking was rated the best experience among three modes of mobility. A bicycle ride happened at a moderate speed, so it was less affected by the latency and inaccuracy of GPS. When the user spent less effort in perceiving the audio, he could better blend himself into the environment, and that led to a smooth and more connected user experience.

Please download my doctoral thesis here.