Richard Rodkin

Richard Rodkin

Creative Coder / Composer. Founder, Memetic Arts. Creator, U-GRUVE AR.

Follow Us

Latest Posts

For the Record

U-GRUVE AR: Turning Environments Into Instruments

As this is our DevBlog’s inaugural post I thought it fitting to provide some initial context before we dive into the depths of technical discussions outright. There are a number of key concepts that define u-gruve, and since the platform is largely unheard of, now seems like the right time to provide an overview.

In short, these concepts are: Audio Augmented Reality, or the ability to provide a digitally mediated layer of information within a real-world context; Music Discovery, which integrates acts of exploration and improvisation into the listening experience; Remix Culture, which embraces the rearrangement and recombination of pre-existing elements; the Rise of the Citizen Performer, in which the normally passive listener finds himself crafting his own musical experience; Indeterminacy, in that the sounds within an environment will never be triggered and controlled in the same way twice; and finally, and most importantly, that u-gruve is not just an app, but an entire music delivery system.

Audio Augmented Reality

Augmented Reality (AR) has largely been thought of as a visual medium, in which a mobile device — either hand-held or head-mounted — weaves an additional, virtual layer of contextual information into our real-world experience, consisting of either 2D text-and-image display, or 3D imagery that appears to have been inserted directly into our surroundings. u-gruve employs the same principle, but in the form of audio-musical information. By assigning this information — in u-gruve’s case, audio clips — to GPS coordinates, we can deliver multiple tracks of sample-accurate, synchronized audio to smart devices as they intersect with those coordinates, enabling participants to mix tracks based on their location.

Music Discovery

When we speak of “music discovery” in the music business, it is usually in the context of understanding what channels consumers use to find new music. In the case of u-gruve however, “music discovery” takes on an entirely new meaning.

Once we have sonified an environment — that is, mapped audio clips and their respective triggers to zones and objects — we’ve effectively created a minefield of musical experience. When the participant initially “trips” one of these musical mines, be it a rhythm, a texture, or a melody, they become engaged and want to hear more, at which point they embark on a mission to find the rest of the piece.

During the Renaissance, the term “Ricercare”, meaning “to search out”, was applied to an entire musical form. A predecessor to the Fugue (whose name also implies travel and movement), Ricercare required the performer to take a small passage of pre-composed music and weave it into a fully developed piece through an improvisational “searching out” of the original germ idea.

We feel that by enabling participants to experience music by virtue of exploration and discovery, they will feel more intrinsically tied to not only the music, but the place as well. We already create strong associations to places via the songs we’ve heard at specific moments; this added dimension only serves to amplify those connections.

Ultimately, this heightened degree of interaction stands to catalyze a fundamental shift in the way music is produced and consumed, as it empowers an otherwise passive listener to improvise and create, and leads to the Rise of the Citizen Performer.

Remix Culture and the Rise of the Citizen Performer

Remix Culture embraces the act of crafting derivative works from pre-existing material, a practice that has driven several large sectors of the Music Industry (Rap, House, EDM, IDM to name a few). It has become increasingly commonplace for artists to publish sets of audio clips for fans to remix, either for fun or in conjunction with a sponsored contest. But where remixing has been limited to those with specialized hardware and/or software, u-gruve opens up access to anyone with a mobile device and the time to peruse a park or plaza. With the line between audience and performer now significantly blurred, and the realization that they suddenly have the ability to create satisfying, emotionally relevant music, a new breed of performer is emerging, in much the same way that the Citizen Journalist was born from the advent of the camera-phone. The era of the Citizen Performer has arrived.


“Indeterminate”, or “Aleatoric” music, a genre established in the mid-1960’s by Earle Brown, John Cage, et. al, is conceptually the crux of the u-gruve platform. The basic premise is that an array of variables, often derived from chance procedures such as casting dice, are introduced into the performance process so that no two performances are executed in exactly the same way. With u-gruve, the participant’s path is the “X” factor, as there is no way of knowing what regions they will cross and when, or how long they will linger within an area. Other factors, such as weather, time-of-day, heading, speed, and orientation could (and will!) also be used as inputs. Each rendering, therefore, is unique. This also poses significant challenges to the composer, as all of the constituent parts need to fit together and work musically at any given time, in any given combination. And aside from the technical mastery that this requires, the composer must also be willing to forfeit a certain degree of control over how the parts come together as a whole.

It’s Not Just an App, It’s a System

Lastly, I’d like to stress that u-gruve is not simply an app containing a few pieces of experimentally-constructed music, but rather a system designed to deliver and establish this emerging format. By its very design, u-gruve consists of original, site-specific compositions, representing a range of styles, genres, and emotive qualities, created and contributed by a growing community of composers, through Studio-M, Memetic Arts’ content development group. As one might expect, each composer’s work brings a fresh and unique interpretation of the selected locations, with the contrasting perspectives adding significant value to the overall appreciation of how physical spaces impact our emotions.

The plan is to continually build the catalog over time to include dozens, and eventually hundreds of locations with associated pieces of music — to enact the widespread sonification of our public spaces, if you will. In effect, it is a new channel for the public to acquire and enjoy music, as well as a new channel for composers and artists to monetize their work.

And of course, there is the social aspect as well, though that is probably best left to an entirely separate post. Ultimately, we are working to effect a large-scale shift in the public’s view of what music can be, and how it can have emotional impact on our lives in the context of living them, while at the same time transforming composers into collaborators, listeners into performers, and environments into instruments.

Leave a Reply

Your email address will not be published. Required fields are marked *