Opinion: Virtual Reality is an entirely new artistic medium in which sounds plays a surprisingly important role
By Enda Bates, Trinity College Dublin
There has been a lot of hype about Virtual Reality (VR) in recent years, but much of this discussion has been about the technology behind the medium, rather than the content itself. This may be about to change however with recent events suggesting that VR is starting to be viewed as more than just a gimmick and perhaps, as an entirely new artistic medium.
While the internet has utterly changed the business and distribution of music and television, the content itself has remained largely the same. With its own language and its own intriguing creative possibilities, VR is something new. In terms of its production from a practical perspective, it often resembles film in its use of video cameras and/or digital imagery. However, in its staging and approach to narrative, it is perhaps more akin to theatre, while also incorporating the interactivity of gaming.
In truth, we don’t yet know how VR will develop, but the increasing engagement with the medium by high profile directors, musicians and artists suggest that the potential creative possibilities of VR are significant.
One aspect of VR that is fundamentally different from other mediums is the importance of sound, and particularly spatial sound. Cinema surround sound has been around for many decades, and indeed the spatial audio technology used in VR is largely a combination of techniques developed in the 1930s and 1970s. While surround sound certainly enhances the cinema experience, it is also true that many films translate just fine to the small screen and pair of stereo loudspeakers.
In this medium, spatial sound can therefore be seen as an extra. While it’s an extremely worthy one in some cases (the spatial sound design and soundtrack to Alfonso Cuarón’s Gravity comes to mind), it’s not a necessity. In that respect, as in others, VR is fundamentally different. Just as in reality, it is ultimately sound that provides a sense of immersion within a scene, whether it’s the real world around us, or indeed a virtual one created using a head mounted display.
Most discussions of VR tend to focus on the visual, but the role of sound is critically important
Right now, as I sit in my living room writing this article, my vision is focused on the screen in front of me and, more often than I would like, the view from my window as I procrastinate instead of type. When I stop and listen however, my perception expands to include the soft chatter of the neighbours in the upstairs apartment, the dull roar of traffic on the road outside and below, the crying of gulls above, and the occasional drip of a tap from the kitchen behind me. It is these sounds that immerse and situate me in the surrounding world, whether they are the focus of my attention or not.
Most discussions of VR tend to focus on the visual, whether it’s the screens and headset, or the imagery captured with a 360 camera or created in a computer. But the role of sound in VR is often overlooked despite it being critically important both in creating an immersive experience and in guiding the viewer’s attention through the narrative.
We need your consent to load this SoundCloud contentWe use SoundCloud to manage extra content that can set cookies on your device and collect data about your activity. Please review their details and accept them to load the content.Manage Preferences
The fundamentally interactive nature of VR distinguishes it from other mediums such as cinema, yet this is also a significant challenge when telling a story. In cinema, the director is in complete control of the frame and indeed much of the language of cinema is based around editing different shots to create a narrative. However in VR, the audience can at a minimum look around the scene (as with 360 video), and in other cases move within the scene too (in the case of computer generated imagery but also soon in video content as well).
So, how can we tell a story if we don’t know if the viewer is even looking at the action, or indeed even in the same room? Well, one answer is sound, or more specifically the spatial aspect of sound. In real life, if we hear a noise behind us, we will instinctively turn around to look in that direction. This is something that directors can use to guide the audience through a narrative in VR (much can be learned from the inherently interactive world of gaming here also). Spatial sound is therefore important in the medium in a highly functional sense, and is far more than a nice, yet largely optional extra like cinema surround sound or surround remixes of classic albums.
One critically acclaimed piece that beautifully illustrated the increased importance of sound is the VR experience Notes on Blindness from 2016, which was released to accompany the award winning documentary film of the same name. The film is based on the audio diaries of writer and theologian John Hull, which documented the author’s experience of becoming totally blind after many years of deteriorating vision. Counter intuitively, the VR app portrayed this experience using a combination of shadowy imagery and most notably spatial sound to evocatively portray the experience of navigating the world without sight in a highly affective and engaging way.
We need your consent to load this YouTube contentWe use YouTube to manage extra content that can set cookies on your device and collect data about your activity. Please review their details and accept them to load the content.Manage Preferences
As VR develops into a new artistic medium in its own right, it will presumably eventually produce it’s own Citizen Kane, it’s Waiting for Godot, or it’s Journey or Shadow of the Colossus (in truth, it will most likely be some combination of all of the above). Regardless of what form that classic work of VR will take, spatial sound will undoubtedly play an intrinsic, yet invisible role.
We need your consent to load this YouTube contentWe use YouTube to manage extra content that can set cookies on your device and collect data about your activity. Please review their details and accept them to load the content.Manage Preferences
As part of this year's Audi Dublin International Film Festival, Immersive Stories: Conference is a two-day event at Dublin's Mansion House on February 24th and 25th featuring world-class experts in virtual and augmented reality fields.
Dr Enda Bates is a lecturer in spatial audio and music composition and deputy director of the Music & Media Technology programme in the School of Engineering, Trinity College Dublin. He is a composer of spatial music, and performs with various groups such as The Spook of the Thirteenth Lock
The views expressed here are those of the author and do not represent or reflect the views of RTÉ
