This was an invited TEDX style talk, given on 22 May 2018 at Queen’s Film Theater (QFT) for the 4th Industrial Revolution Challenge on VR and the Arts 4IRC VR challenge.
Please refer to the text alongside the slides:
I asked the audience to close their eyes for about 1 Minute and to intently listen.
Talk:
“I have asked you to focus on the sounds in the room around you for a moment. I find listening in such a focused and intense way very relaxing but it can also be a kind of meditation, esp. as we close our eyes our ears become more attuned.
As a musician (I play the saxophone), listening is very much the main activity I engage in so it is not surprising I asked you to listen a little more intensely for a moment, and one reason for this is that listening is often taken for granted – not only because it’s a sense that we cannot naturally switch off – many current VR experiences are designed first and foremost as highly visual spaces, leaving sound as an addendum.
Sound, and indeed listening, in many of the current VR spaces, tends to take second place, but sound is essential to guiding us as human beings. Sound is 360 degree. We are so obsessed by vision that we tend to neglect listening as we focus on the seen world. The reason for turning around when you hear a creaking door or a voice behind you is sound, not vision. So sound should be a much more essential and well designed component in VR.
However, the design for high quality, immersive audio is still in its infancy in VR – and we might want to remind ourselves that VR itself is still considered a new, or at least emerging medium, although as Reuters agency has predicted AR / VR is expected to grow into a $95 billion market by 2025. The market analyst ‘International Data Corporation (IDC)’ even forecast that AR/VR spending will accelerate over the next several years so as to achieve a compound annual growth rate (CAGR) of 198.0% over the 2015-2020 forecast period and totalling $143.3 billion in 2020.
What is particularly exciting to someone like myself working in the Creative Industries is that the strongest demand for this technology currently comes from industries in the creative economy – specifically, gaming, live events or video entertainment (see SLIDE 2).
But let me be clear that by pointing to a more focused engagement with sound, I am not criticising the power that pictures play in story telling in VR. We are narrative beings; we have always told stories and recently scientists even suggested that the art of story telling is linked to evolution, that it may even keep us alive.
When researchers from University College London turned their attention to the Agta (see SLIDE 3) – an indigenous people who live in scattered, isolated mountainous parts in the Philippines – they determined the power and purpose of storytelling and one of the main findings was that good story tellers improve their chances of being chosen as social partners but also that they have healthier offspring.
So, there is a strong connection between storytelling and evolution.
And of course story telling is also very much about listening (sitting around the campfire listening to each others’ stories..), and when listening is such central part for us as human beings, for being with each other, I ask myself why listening is not attended to with the same care and consideration in VR spaces as the visual story line is cared for? This is particularly amazing as listening is 360 (sound is always all around us; it is spatial) – and thus might actually be one of our original, the real immersive sense, whereas vision – for hundreds of years – has been flat; it has been there, out in front of us.
The other parallel question I ask myself is How is this medium, this Virtual Reality is potent or, brave, vigorous? I am taking the cue here from the word’s Latin origin ‘virtus’ / virtue which has to do with vigor, moral strength, bravery courage, and also has a hint of ‘manliness’ (as vir stands for ‘man’).
So, if virtual (virtus) stands for being brave, excellent, potent, manly, I have to admit that from where I am standing it looks often the exact opposite’; it can look awkward, clunky and at its worst reminds me a bit of this (see SLIDE 4).
But more so, the headsets / goggles actually box us into something quite small, our multi dimensional, multi-sensorial world become narrow and despite the fact that we might feel immersed, or experience something previously unseen (jumping out of a plane, climbing Mount Everest or being on a rollercoaster ride…), physically we can become rather absent/detached; even vulnerable, and at times maybe look even a little stupid (see SLIDE 5) -:)
By shutting ourselves into the virtual world, our bodies can become quite vulnerable, trapped even. Maybe this is what being a ‘digital captive’ actually means? We don’t hear what’s going on around us nor do we have that same tactile feeling when someone enters or leaves the room. We don’t know what others around us are doing or what they might be doing to us. And it is here where I am coming back to the importance of sound. As I said sound guides us in orienting ourselves in the world and thus deserves a more essential place in VR environments.
So, will these immersive spaces transform the arts?
We could easily say ‘yes of course it will’ (the numbers I mentioned seem to suggest it!), but I would like to re-phrase the question a little, as what we should be asking is whether we, as designers of VR and as designers of many other things that characterise the so called 4th Industrial Revolution – think: ubiquitous, mobile supercomputing. Intelligent robots. think: self-driving cars. neuro-technological brain enhancements. genetic editing – whether we are in the process of replacing ourselves as humans, potentially turning us into something previously unimagined, something quite spectacularly different, something that fuses us humans much more intricately with the digital and the biological, and indeed in this process challenging ideas about what it means to be human – a question often asked in this process of 4th Industrial Revolution.
The man often mentioned as having coined the term The Fourth Industrial Revolution, Founder and Executive Chairman of the World Economic Forum, Professor Klaus Schwab has argued that we are at the beginning of a revolution that is fundamentally changing the way we live, work and indeed relate to one another.
If we look at the current state of VR I would argue that we are still somewhere at the threshold; in-between the 3rd and 4th IR, maybe we are slowly moving towards a so called 4th IR state, but we have a long way to go if that’s the way we want to go. At present VR is still strapped on as the metaphorical gas mask, but we are moving towards embedded or more embodied experiences.
Explorations of haptic feedback systems, vibrotactile /e-textiles, olfactory elements are under way.
From a performer’s point of view, and specifically from a musicians’ way of listening, I would like to see concepts from performance (embodiment, gestures, breath, tactile feedback, acoustic perception) exploited in more creative ways so that we move many current and very excellent VR experiences – but that, as I have argued, are still driven by visual metaphors – into a different space.
See SLIDE 6:
Gabo Arora, founder and president of Lightshed (a virtual reality and social impact start-up) who co-directed with film producer Chris Milk, the virtual reality film, Clouds Over Sidra, (for the United Nations); the VR film that put you inside a Syrian refugee camp and follows a day in the life of 12-year-old Sidra, a girl who has lived there for 18 months with thousands of other refugees – he said something about how this emerging medium allows us to discover a new grammar of storytelling and emotions (VR as enhancing empathy).
I think this way of thinking is really essential, and that rather than conceiving of technologies as extensions of our logical human practise and action (they are archives of techniques, so Jonathan Sterne argues), where our human action of walking leads to VR experiences of walking up mountains, or our actions of jumping and falling leads to the VR parachute experience, and so on… we need to think beyond the question of whether VR will transform the arts, but more so about what VR hasn’t done yet and what this new grammar of VR is.
With a team of designers (sonic artists and visual designers) I am currently finishing a VR piece called EMBRACE – a work about music making and disability and sound and listening were the shaping factors in making this VR piece.
Let me summarise. See SLIDE 7:
When we think beyond technologies as extension of human action, and embrace a design process that moves along from a visually centred approach and that takes into account how other forms and elements (breathing and of course listening as I have focused on in my talk); how such elements might inform the future design of VR and what this new grammar of VR is.
These are the essential questions for me in shaping the next generation of VR experiences.
In the spirit of my argument on listening, I truly Thank you for listening.
2018 Copyright: Franziska Schroeder
Performance Without Barriers 