Geographies of Interactive Digital Narratives

This post provides an account of the Geographies of Interactive Digital Narratives session that took place at the Royal Geographical Society (with Institute of British Geographers) Annual International Conference on 29^th August 2019. The session was co-organised by myself and Dr. Scott Palmer from the University of Leeds, and was gratefully sponsored by the Digital Geographies Research Group of the RGS-IBG.

____________________________

Introduction

The turn towards the digital within the humanities and social sciences over the past couple of decades, recognising the increasing pervasiveness of computational hardware and software in everyday life and societal organisation, has brought together scholars and creative practitioners from a wide range of disciplines, seeking to investigate and codify exactly what is distinct about the ways in which digital technologies mediate our lives.

Yet it is apparent that event amidst the diversity of digital communication tools in use today, humans still rely on one of the oldest and most widespread methods for sharing information and experiences: storytelling.

Not only do digital media provide platforms for communicating existing narrative forms, such as e-books for novels and online video streaming sites for film, but they enable new methods of storytelling through their computational affordances. While video games have proved the most influential of these computational narrative media in popular culture, comprising an industry in which 2.5 billion players now spend a total of $152.1 billion on games globally, digital storytelling has embraced a wide range of forms. These have included hypertexts – branching narratives communicated through the user’s navigation of hyperlinks; immersive theatre that uses technology to augment performance spaces; and interactive TV and film – screenplays whose narrative path depends on the inputted choices of the viewer, the most well-known example being last year’s Black Mirror episode Bandersnatch.

Even within the area of ‘games’ itself there is a huge amount of diversity, particularly as the internet, mobile and locative technologies have enabled more pervasive forms of play in which game narratives can bleed into our everyday relationships, routines and environments (see, for example, alternate-reality games and location-based games).

With emerging narrative uses of digital technology being recognised by public sector and creative industry members as the driving force behind the so-called ‘immersive economy’, we are also seeing increased investment into the production of creative works that use virtual reality (VR), augmented reality (AR) and mixed-reality (MR) technologies. In the UK, this has filtered through to academic and arts council funding too, as government bodies seek to make the UK ‘world leaders’ in this burgeoning ‘immersive’ sector.

But why should this topic be discussed at a geography conference?

What many narrative theorists working in the area of digital media have observed is that it is digital media’s unique ability to represent navigable space that is essential to the dramatic power that digital narratives have. Whether it is movement through branching links or choices of hypertexts and interactive film, physical environments aided with GPS and digital mapping, or designed 2D and 3D environments, navigation is typically central to the flow of narratives in digitally-represented spaces.

Beyond how digital narratives are experienced, there are also geographies to how these works are produced and consumed, which often differ and compare interestingly to more traditional narrative media. From a scholarly perspective, this digital evolution in narrative form, production and consumption might also point towards new methodological opportunities and challenges in our work as researchers.

Even given these observations, there has been minimal consideration of the narrative significance of digital technologies within geography, despite a long history of geographers engaging with other narrative forms (particularly in the prominent sub-field of literary geographies). At the same time, in other disciplines where research on digital narratives is undertaken, such as game studies, there has been a clear lack of critical engagement with geographical conceptions of space and place.

What this session aimed to do was to fruitfully bring together narrative scholars and practitioners from across the arts and humanities to provoke new directions of scholarship between geography, narratology and the diverse range of related disciplines; to grapple with the implications of interactive digital narratives for how we understand space and place, and discuss the conceptual and methodological opportunities and challenges that are provoked by research and creative practice in this field.

Duncan Speakman – ‘It Must Have Been Dark by Then’: Geographic Narratives and Temporal Composition in an Interactive Mobile Audio Artwork

Our first presenter was Duncan Speakman from the Pervasive Media Studio in Bristol, who discussed his locative audiowalk and companion physical book ‘It Must Have Been Dark By Then’. This work uses evocative music, narration and field recording to communicate stories from environments that are changing – from the swamplands of Louisiana, to empty Latvian villages and the edge of the Tunisian Sahara.

Rather than being a site-specific work with a set route, users are asked to seek out particular types of spaces in their environment, with the software building a unique map for each person’s experience. By choosing their own paths through the city, participants can explore ideas of environmental change in the places they inhabit and consider how they might connect to locations elsewhere in the world.

As well as being directed by where participants choose to walk, the app occasionally asks you to reach a certain nearby location, indicated by the blue dot and basic directional markers on the app’s blank map. Duncan explained that this part of the work aims to draw attention to the material attributes of participants’ surroundings, with any challenges in reaching these locations provoking an acute awareness of the physical, legal and social boundaries that constitute their present environments.         

Once a location is reached, participants read the associated chapter of the book while listening to the atmospheric sound and field recording from the location the story derives from. Here, Duncan’s work seeks to draw on the immersive capacity of audio and books to mentally transport you to different locations, despite being very much embedded in a particular place and time at the moment of listening and reading.

Throughout the walk, the app tracks the locations the user visited. Then, in the second half, participants are invited to walk back through the sites they visited previously, listening to narration that continues the stories that were read about in their specific locations. As participants remember reading about individual experiences of environmental change at the sites they visited, the route of their walk becomes a kind of physical sculpture of memory that draws together near and far, past, present and future in the singular moment of inhabiting a particular place at a particular time.

Indeed, temporality was central to Duncan’s thinking when designing It Must Have Been Dark By Then. While most locative media involve participants heading to particular locations to receive content, Duncan believes that the gap between these points is often more important to the composition of the story. By purposely cultivating slow and absorbing methods of communicating the stories, Duncan aimed to create impressions of environmental change that move beyond paradigms of the ‘eco-sublime’ perpetuated, for example, by images of collapsing glaciers and ice sheets. Instead, Duncan argues, we need to look at the everyday environments around us and think about their timelines.

Ultimately, It Must Have Been Dark By Then seeks to use a unique form of interactive storytelling to develop new forms of attention towards the environments we live in – beyond pervasive and simplified divisions between subject and object, human and nature – to articulate methods of witnessing that trace these entanglements; imagining the world not as for humans, but with humans.

Jonathan Barbara – Narratively Consistent Virtual Reality Interactive Narratives in Cultural Heritage Experiences

Our second presenter was Jonathan Barbara from the Saint Martin’s Institute of Higher Education in Malta, who discussed the potential of interactive digital narratives for telling stories of life at cultural heritage sites; bridging between tangible cultural heritage (artefacts with a physical presence) and intangible cultural heritage (cultural practices, traditions and representations that have no lasting physical presence).

Jonathan began by outlining the importance of digital technology for cultural heritage today. While the promotion of digital cultural heritage is certainly not new, particularly in museum spaces where technologies such as touchscreens are now prevalent, VR and other immersive media formats have the potential to expand accessibility while improving the safety of historic artefacts and enabling engaging and sensual forms of learning. The significance that digital manifestations of heritage assets can have today was illustrated by the example of how models of Notre Dame from the Assassin’s Creed video games have been used to help guide the Paris cathedral’s reconstruction effort, following the fire that took place there earlier this year.

What Jonathan is particularly keen to explore is how the mechanics of interaction with digitally-mediated cultural heritage might be informed by the specific geographies of historic sites – the built environment and cultural practices that occur there today and have occurred there in the past.

With this aim in mind, Jonathan ran a workshop for students that asked them to design ideas for a casual game relating to the Hypogeum in Malta, a prehistoric subterranean site thought to be a sanctuary and later a burial site. The workshop participants were given a map of the Hypogeum, alongside basic textual and visual prompts such as photographs and detailed descriptions relating to life at the site. No guidance was given to the students as to what kind of interactive experience to create.

Interestingly, Jonathan found that his students’ game designs consistently returned to tried-and-tested interaction mechanics from popular culture; notably escape rooms, Pokémon GO and treasure hunts. This is despite there being a wealth of behaviours, rituals and systems associated with the site upon which interaction mechanics could potentially be based, including acts of digging, burial rites, and the unique social fabrics that were present at the site.

Jonathan concluded by suggesting that interactive storytelling mechanics that are more deliberately inspired by the specific geographies of heritage sites might help visitors to better make sense of their repositories of tangible and intangible heritage, and called on those in the session to consider and discuss how this might be achieved.

Lissa Holloway-Attaway – Play, Place, and Telling Micro-Histories through AR Storytelling Experiences for Children

Our third speaker was Lissa Holloway-Attaway, an Associate Professor of Media Arts, Aesthetics and Narration from the University of Skövde in Sweden. Lissa’s presentation discussed the AR playable book series called KLUB (Kira and Luppe’s Beastiarium) that her and university colleagues have created to engage children with folklore from the Skaraborg region of Sweden.

The KLUB series revolves around the characters Kira and Luppe, a young werewolf and vampire, and their struggle against a ringleader character who attempts to make them perform in a travelling circus. It is in Kira and Luppe’s journeys with the circus where they encounter fantastical beasts, with each book in the series telling an individual story based on folklore from one county in Skaraborg. When children, accompanied by their parents, read the books and scan certain images within them using the KLUB app, the characters portrayed in the images appear as 2D and 3D visualisations that are then collected in the app’s ‘beastiarium’.

This form of interactive storytelling is further reinforced by other methods of cultural engagement that Lissa’s team have developed. Having formed partnerships with a range of local, national and international collaborators, characters from the KLUB books are now appearing in numerous local museums in Skaraborg as part of playable and augmented-reality exhibition experiences. Through this broad range of interactive and site-specific activities, the creators are aiming to use technology to foster connections between tangible and intangible heritage, employing the fantastical themes of the stories to encourage families to explore the diverse landscapes of the Skaraborg region in person. Working further towards this aim, the team plans to extend the KLUB universe into other media forms, including board games and guidebooks.

Beyond the KLUB project, Lissa is interested in how geographers and artists might work together to find ways of re-imagining landscapes through the medium of maps. To this end, Lissa has been involved in a recent project that engages children with their local cultural heritage sites by asking participants to map these places in the game world of Minecraft. Given the representational and abstract qualities of maps, Lissa made the point that this type of exercise benefits from being both creative and actively informed by the geographies of these individual sites. Indeed, these examples, Lissa suggested, point towards a wider trend of using the interactive possibilities of digital technology to enable experiential engagement with place.

It must be noted that, during the session, we discovered that one of our audience members was from Sweden, and that her family have actively used the KLUB books to teach the children about their heritage in Skaraborg since moving away from the region. Lissa explained that she has heard of many similar experiences, and that one of the aims of the project was to create a fairytale for whole families to explore together, connecting local heritage to the personal ties of ancestry and memory.

Jack Lowe – Environmental Storytelling: A Digital Frontier for Narrative Geography

I was the final presenter in our session, exploring the relationship between navigation and narrative in exploration-based video games by discussing a set of game design practices known as environmental storytelling. I went on to consider what potential environmental storytelling might have for interactive digital narratives that operate in physical settings using locative, mobile and ‘immersive’ media, explaining how environmental storytelling can provide a conceptual and practical toolkit for making sense of relationships between geography, narratives and digital technology. (A full written version of this presentation is available here).

I began my presentation by outlining how the ways that we interact with digital media are characteristically navigational – whether it’s in our use of hyperlinks to travel between websites, use of geolocation to traverse physical space, or deliberate movement through the rendered worlds of video games. What is distinct about interactive digital narratives is how these processes of navigation are shaped into the dramatic enactment of plots.

In the case of environmental storytelling in video games, developers use evocative narrative elements in the form of in-game objects (e.g. written notes and artefacts), sounds (e.g. music, ambient sound), haptic feedback (e.g. how the avatar moves) and more to embed story information into their worlds for players to find. This distributed composition of the storyworld – as opposed to a consistent narration – not only allows for the  communication of a range of character viewpoints (multi-vocality), but also means that players can direct where to focus their attention in the world, and the extent to which they do so (dynamic focalization). The result is a highly personalised account that is closely tied to the player’s own interpretations and domains of meaning-making.                                  

Nonetheless, game developers can also employ subtle design techniques to guide the player’s navigation. These are gating, in which entry and exit points are coordinated to structure the player’s exposure to information; signposting, in which light and sound directs the player’s attention towards significant narrative elements; and pacing, in which the spacing between narrative information is managed to elicit dramatic tension and mood. Taken altogether, environmental storytelling operates through the curation of a mise-en-sensibilité of player sensibilities, material hardware and aesthetic representations, through which players assemble meaningful experiences within a flexible structure that enables their navigation through the game world to unfold in dramatic and emotionally engaging ways.

With the advent of more widespread adoption of locative media, 360° video and AR/MR technologies, artists are increasingly invoking navigation in ‘real world’ places as the basis of interactive storytelling. Yet many of these hybrid ‘immersive’ media present challenges that aren’t typically faced by video game designers, particularly concerning the contingency of embedding narrative elements in living, changing environments; the often uncomfortable intersection of the gameplay with existing material processes, histories and socio-legal norms; and retaining dramatic agency when the boundaries between the storyworld and everyday life are less clearly defined.

Particularly when these challenges and affordances are confronted through practice-based research, as part of a creative process, this is why I concluded my presentation by arguing that environmental storytelling as a concept can be useful for scholarship beyond just video game design, providing opportunities for apprehending the broader ecology of materials, bodies, social norms and physical processes that shape how digital narratives are produced. In so doing, environmental storytelling can potentially offer an important frontier for interdisciplinary research across the digital humanities into how the relationship between navigation and narrative plays out across a range of contexts.              

Panel discussion

The first main topic deliberated in the panel discussion concerned the relationships between physical objects and digital representations in interactive digital narratives, and what the materiality of objects used in these storytelling media adds or takes away from participants’ experiences. One common theme across all the responses was the power that material culture can have for transmitting stories across spaces and times. Duncan notably emphasised that his decision to include a physical book as part of It Must Have Been Dark By Then was to incorporate the stories’ words into the processes of aging and decay being experienced by the environments that are both represented in the text, and in the physical sites that participants travel through. Later, Lissa reiterated that Skaraborg’s physical environment was always one of the starting points for her team’s work, and the fictional frame offered by the KLUB books is aiming to heighten engagement with the material culture and built environments that people can interact with in person.

Another key pivot of our discussion centred on how design processes for interactive digital narratives are managed. The questioner was considering the predicament Jonathan described in his presentation, regarding how the mechanics of interactive digital narratives might be informed by the geographies of their sites, and suggested that part of problem may have been that people think about digital media such as games in terms of genres rather than affordances. Consequently, perhaps there is a need to embrace a more bottom-up design process as opposed to a top-down implementation.

In response to this comment, Duncan made the point that allowing large numbers of voices into a creative discussion can sometimes hinder innovation. All of the presenters concurred that focus groups in particular can be difficult spaces to enable creativity due to the saturation of diverse viewpoints, and the groupthink that can ultimately occur when certain voices are heard and others are silenced.

Our final question asked all of the presenters to consider what is left out of the various narrative media we discussed in the session. In my response, I explained how many exploration-based video games are based on worlds that are devoid of characters you can interact with directly. Instead, the narrative structure of these games often follows a treasure-hunting or evidence-gathering theme in which the player is able to travel and search almost unhindered through the world. Not only does this format lend itself to particular types of stories at the expense of others (e.g. detective narratives) but it can represent an uncomplicated portrayal of the power relations that condition our capacities to navigate through environments in everyday life.

In Jonathan’s answer to this question, he highlighted the challenges of adequately replicating the specific lighting and sound conditions during rituals in places like the Hypogeum, thousands of years in the past. While having no description or depictions of these prehistoric rituals leaves much to the imagination, it is possible to simulate the entry of sunlight and capture the sound profile of underground cave reverberations, which may reflect some of the ambiance of the site. Nonetheless, even with the technology available today, this task remains a computational challenge.

______________________________

As one of the session organisers, I would like to thank everybody who attended Geographies of Interactive Digital Narratives and contributed so thoughtfully to the panel discussion. I’m particularly grateful to Scott Palmer for his support in convening this session with me, the RGS-IBG for their help and quick responses when administering the session, and the Digital Geographies Research Group for their sponsorship. Lastly, I want to thank my fellow presenters Duncan Speakman, Jonathan Barbara and Lissa Holloway-Attaway for sharing work that engaged so insightfully with the session theme, alongside everybody who responded to the original call for papers. I hope the topics and debates explored in this session will continue to remain on the agenda in Geography and across related disciplines, in line with the increasing influence of digital narratives across our societies and cultures.