Extending Realities, Expanding Readings? Spatial Reconfigurations and Activated Presence in XR Comics’ Experiences

By Carolina Martins

Virtual space, a simple pretext for a nomenclature. But you don’t even need to close your eyes for the space evoked by these words, a dictionary space only, a paper space, to become alive, to be populated, to be filled.

-Georges Perec, Species of Spaces (13).

Extended Reality (XR) encompasses Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) programs and technologies. However, none of these technologies, despite their digitally engaged nomenclature, is devoid of anchors to the physical world. Even when simulating (or proposing) an immersive and interactive environment beyond our actual reality, these composed and alternative realities still need a set of physical hardware devices to suggest enhanced interactions. Although some may say that space gains form because of its own limits, we mainly relate to a perception of space as something abstract. That is, although we can imagine spaces, we can hardly imagine them without any kind of boundary because, even when picturing a vast landscape, there are limits to what our mind’s eye is seeing.

Therefore, the use of the term “XR” (extended reality) in this article does not necessarily refer to a unique experience where different types of digital realities are functioning at once, but rather to the set of experiences proposed by Marc-Antoine Mathieu’s S.E.N.S^[1] (2014), a graphic novel that begins as a book and metamorphoses into an art installation and a VR game^[2]. I have decided to use the term XR as an almost playful provocation, in the sense that I consider that it is possible to create augmented or extended realities with or without resorting to VR or AR apps and devices. Like Perec, I believe we “don’t even need to close our eyes for the space … to become alive” (13).

It has been a while since artists started dealing with the possibilities of using actual/physical 3D spaces in relation to the audio and visual media at their disposal. Be it through the placement of viewers inside video art installations or inside other dynamic environments where they can interact in a participatory way, the search seems to reveal one and the same goal − one that relates to humanity’s constant desire for transcendence and release from the constraints of real places: the extended space experience. And comics are no exception in this respect.

I consider that, by making use of the gallery as a type of ‘augmented space’ (cf. Manovich) for writing and reading, installations propose their own constructed space as an ‘augmented reading’ device in itself. By putting together the characteristics of a structured narrative composition and the actual space of the gallery, such installations conduct the reader to access and expand discursive levels. Since the gallery can turn into a form of augmented space, so too can the reading that it mediates becomes augmented through the expansion proposed by its use, opening possibilities and mixing realities. Manovich’s abstract for his article “The Poetics of Augmented Space” (2006) argues that “augmentation is reconceptualized as an idea and cultural and aesthetic practice rather than as technology.” Although his notion of augmentation implies “immaterial layers” (220) over the real space, I consider how augmentation may be achieved by analog and digital media alike. A great example of this is Olafur Eliasson’s The Weather Project (2003-2004), wherein the mix of internal and external environments as an artwork becomes complete when inhabited by an audience that does not need more realities besides that one.

In Inside the White Cube (1986), Brian O’Doherty also acknowledges the gallery’s capacity for being altered by the installation structure, “itself becom[ing], like the picture plane, a transforming force” (45) and promoting the “effect of the container on the contained” (69). The gallery can go from being a container that displays works of art to a chamber that can become an immersive landscape, itself a constitutive part of the piece, no longer delivering a mere space for presentation but rather becoming readable by means of the use of its own characteristics to promote communication (simulated or not). By being simultaneously actual and fictional, the architectural space works like a virtual environment on its own.

I would like to take advantage of this point to reinforce that my take on realities implies a distinction between actual and virtual, but it does not deny the fact that VR and AR are indeed realities, even though they are grounded in actual/physical devices. A long road in our understanding of comics’ potential to grow off the page has been walkedsince Scott McCloud’s first reference to an “infinite canvas” in Reinventing Comics (2000). Even Paul Gravett’s “Hypercomics: The Shape of Comics to Come” collective exhibition held at the Pump House Gallery (London) in 2010 seems far away when we think about the evolution of the various realities’ characteristics today. With a somewhat provocative title, and featuring, besides others, Daniel Goodbrey’s The Archivist and Dave McKean’s The Rut, the exhibition essentially refers to the multicursality of comics, which relates to the digital comics that were spreading through several digital platforms. Goodbrey’s piece has a digital twin that can be read on his website as a hypercomic for the computer screen, functioning with a zooming tool he built himself at the time, the Tarquin Engine.^[3] I refer to Goodbrey’s and McKean’s works because, on top of the fact that both authors often engage with comics in specific spaces, their works for the PumpHouse event raised some questions that, from my perspective, have everything to do with augmented and expanded spaces and how these can be built without accessing VR and AR apps and by embracing the architectural characteristics of the physical space instead.

This article reflects on how the reader connects with works that emphasize comics’ potential to go digital, but especially their potential to interconnect and interact with different media and spaces, as is the case of Marc-Antoine Mathieu’s S.E.N.S in its different forms − book, installation and VR game.^[4] This use of different tools, media, and devices explores different spatial configurations and, consequently, proposes different types of experiences of reading from which arise different meanings. Although departing from the codex, the additional layers conveyed by a set of different formats contribute to extending not only the types of readings, but the narrative itself, along with its meanings.

S.E.N.S was first published as a book and then reconfigured to be presented as a gallery installation and later a VR game, turning each experience into a different way to connect with the same narrative. The narrative gathers layers of meaning with each experience, differing, for instance, from what happens with Stu Campbell’s Modern Polaxis (2016), a bundle composed of a paper book and an AR app. In this case, book and app entangle and expand on each other in order to complete the whole of the narrative. This does not mean that Modern Polaxis cannot be read without the app or that it does not make sense without it, but the reader will not be able to access all of it only by reading the book, since part of the narrative is concealed in the app, which completes the story and gives meaning to its use. With S.E.N.S, however, book, installation, and game do not need each other to tell the whole story. They rely on each other to expand the meanings of the narrative relating to the particularities of the experience. That is, the experience that each medium proposes and enables contributes to a new reading that builds on a previous one (another medium) instead of being necessary to complete the whole of the narrative. Thus, S.E.N.S dwells in both intermediality and transmedialityin the sense that it comprises not only a graphic novel or an installation where comics and animation meet, but also a network that adds up to this a VR game and an exhibition featuring a contemporary art mood where several sculptures are presented.

Marc-Antoine Mathieu recurrently conceives the composition of the book as a medium by taking into account the reader’s inclusion and participation in the world he is building, as is the case with the Julius Corentin Acquesfacques series (2003 − present), wherein the main character is constantly confronted with the reality of being a construct living in a diegetic space. Mathieu makes this approach more subtly inS.E.N.S, which is a kind of a unicursal labyrinth, into which Mathieu drops both character and reader without any notice or orientation. Mathieu built several labyrinths throughout his career, including in the mentioned Acquesfacques series, in Mémoire Morte (2000), and in Labyrinthium (2013).^[5] However, S.E.N.S gets its singularity from the fact that, unlike in the other works, which are usually composed by tight spirals that reflect on the prisoner’s claustrophobia, in S.E.N.S Mathieu takes down the labyrinth’s walls and sets it up in a desert, turning the experience closer to the prison of agoraphobia. This imperceptible labyrinth then turns into an expression of an openness of the graphic novel to other media and spaces, where he could go on exploring the entanglement suggested by the notion of labyrinth but through the complexity of the “Grand Rien,” the Nothingness.

For its co-relation between physical book, physical space, and digital technologies and devices to achieve the whole or to build on it, this case study allows a reading of experiments that begin with comics and reflect their potential to jump off the page. These are the experiences that will allow me to drawon from concepts such as virtual and actual, in conjunction with human geography and spatial experience notions andargue that reader and space(s) of presentation are co-dependent in order to be activated, being simultaneously products and producers of one another (cf. Hayles 29).This article aims to i) emphasize the materiality of the book in its expression as a reading space and the meaning of which depends on, and is produced by, its handling and navigation in relation to an effective tridimensional space and a VR environment; 2) consider how object and reader mutually appear and actualize one another between real and virtual intermittences; and 3) analyze the “augmented experience” of the works in question. During my analysis, I will take into account some notions of comics theory and other relevant issues to discuss the medium’s possibilities for expression and expansion. In the particular case of S.E.N.S, I will do this from a one-panel-per-page comic book to a VR experience.

Handheld Devices I

[V]ision is a conditioned thought, it is born “in virtue” of what happens in the body, it is excited to think for him.

-Maurice Merleau-Ponty, O Olho e o Espírito (34)

S.E.N.S can be summarized in two words: light and limit. Essential to sight, light encloses space in its field of vision. Similar to the experience of reading Martin Vaughn James’s The Cage (1975), in the sense that it is also a one-panel-per-page graphic novel, S.E.N.S invokes a relation between the book space and actual space surrounding the reader. This is possible, in the case of Vaughn James, through the drawings that simulate a first-person point of view (POV) and, in the case of Mathieu, in the mechanisms that are sometimes used to simulate the same POV. When the reader turns the first page of Mathieu’s book, a black page, the action mimics opening a door to the bright desert, something changing within the book that relates to our reception of it. Wherever we happen to be while reading, the virtual spaces of the book (the diegetic ones) and the actual spaces where the readers experience them (non-diegetic), connect, creating a blurred barrier between worlds, which in literary terms would be studied in the figure of metalepsis.^[6]

This is a particular feeling that can come to the reader while confronted with the silent double-spread pages of James’s book. From my perspective, the metaleptic figure is a key figure in the analyses of books with this kind of layout, as well as in other experiences like gallery comics or AR and VR comics. This kind of comics layout allows for a blurring of boundaries, a liminal state, that is similar (at least theoretically), for example, to the experience of a digitally augmented environment. Or, comparable to the experience of reading in an architectural space, where the narrative is set up to be read in connection with that space and its surroundings.^[7]

The connection the reader feels with S.E.N.S through the conjunction of layout choice and narrative theme works as a metaphor, not only of life, but also of the process of consciousness of space and our place in it, which perhaps is also a kind of metaphor for life in itself. While James confronts us with a disjointed landscape of ruins of former inhabitable buildings and with a disruptive relation between text and image, Mathieu surrounds us with a silent spectacle made of a deserted landscape, constantly being intruded upon by elements that defy its whiteness and its stillness. What we gather from this is that the simulation of a limitless space through the first person POV creates a virtual tunnel that leads to a reading of S.E.N.S that considers that,as Brian Massumi writes, “[a] limit is not a boundary. It is open… not unreal. It is virtual. It is reality-giving” (147). Consequently, also considering the book as a virtual object in the sense that Pierre Lévy’s gives it. This is, considering that virtual is “not the opposite of real. [but] … a way of being potent … that promotes the creative processes, opens possibilities and senses in the commonness of the immediate physical presence” (Lévy, 12).^[8] Therefore, the book exists by its multiplicity and by its potentialities that await to be concretized by means of the contact and involvement with the reader. Like a space that is not homogeneous or still, as Doreen Massey (2005) would defend, the work keeps on creating itself through its external relations with readers and spaces.

So, in the labyrinth that S.E.N.S turns out to be, our field of vision trespasses the limits of the page, which end up being expanded by our surroundings, whereas in the installation, our field of vision is tricked by the lack of light inside the gallery, the scenography, and the simulated movement of the videos. On the one hand, our field of vision seems to be limited by the walls and the darkness of the room; on the other, it reveals a sense of expansion simulated by the camouflaged screens. While wandering through the gallery, our field of vision is simultaneously being restricted and expanded. The layers of space do not merely stack up one on top of the other, but rather intertwine meanings that are mediated by the figure of the reader as they go, which might be understood as an analogy for the one-way labyrinth that both character and reader are roaming.

If we take into account Johanna Drucker’s notion of performative materiality (2009), the reader is one of the constitutive elements of the event that is taking place, being responsible for one of the others: the interpretation. These two elements connect with the meaning-making properties of the aesthetic object, not only through the act of looking at it, but also by walking through it. The actions proposed by the work (its haptic experience, for example) and performed by the reader are producers of knowledge, and therefore essential to the reading process and narrative development. So, we must think of “meaning as an emergent phenomenon” (Portela, 2012:39).^[9] The reader uses their physical and cognitive skills to decode the object and, at the same time, the ways by which the object connects the navigational cues to the narrative reveals the reader as an effect of reading (cf. Drucker 6).

Whether book or installation, S.E.N.S’s potential multiplicity is not only due to theinterpretations of its stories, but also by means of what its spatiality suggested to the reader. The physical contact that occurs between reader and work is significant, acting as a means of actualization for both. For instance, the handling of the book invokes the gesture of turning the page, and going through it from page to page contributes to the progression of the narrative while producing meaning. Thus, it is through this physical and cognitive relationship between reader and book that they reveal themselves to one another, as Noe puts it (12). Though the reading process is primarily the successive flipping of the pages as a mechanical gesture that suggests a state of immersion, an interruption takes place when the reader encounters the desert map. This can be considered a formal discontinuity of the book, one that works as a metaphor for the spatial discontinuities taking place throughout the narrative. In this line of thought, I consider S.E.N.S a metaphor for the process of gaining consciousness of space in the particular environment of a unicursal labyrinth without walls, its vastness surpassing the boundaries imposed by each respective interface, be it the page, the wall or, later, the screen.

The interruption creates awareness of a familiar gesture: the unfolding of a map. In this way, although awakening us from a somewhat immersive/automatous state, instead of disrupting that feeling of immersion, the intermission facilitates an awareness of the gesture, contributing to a higher level of self-consciousness that allows the actualization of the reader with regard to the book. In its turn, the book is actualized by the reader’s skills in decoding it, without which the reading would not completely succeed and the book would become a mere object “hidden in plain sight.” The contact with the map grants the reader awareness of the space, turning them into a visitor of the book.

With only an encrypted message, the map mirrors the need for (or lack thereof) orientation in S.E.N.S^[10] The map is not only a key object for the narrative, but also a key subject for the reflection we are engaging with. The map is a “crucial element in our ordering the world, positioning ourselves, and others human and nonhuman, in relation to ourselves.” This is a brief description by Massey (105), who goes on to consider that what space gives us is a “simultaneous heterogeneity,” holding out the “possibility of surprise.”

To look at the map of the desert − of the labyrinth − that comes within the representation of a book found within the physical book also makes us gain consciousness of the space and its relation with the sense of sight. The act of seeing, in Noe’s and O’Reagan’s (2002) view, is an exploratory activity that depends on patterns of interaction between the perceiver and the perceived environment (566). Therefore, when we see an encoded message inscribed in the emptiness of the map, we can consider it a metaphor for a lack of interaction because, although it exists in this desert, it is ephemeral.

In an environment devoid of significant places and which demonstrates a succession of layers to fill the same set, the only points that could be called on to integrate the map would be those that would trace the track of the arrows already followed. However, these appear and disappear between the need to guide and the purpose fulfilled. This is one of the situations that provides a regular awareness of the spaces with which the reader continues to relate. Invoking at the same time the space/place dichotomy, as well as the concepts of “smooth” and “striated” space (Deleuze and Guattari), the characteristics of these spaces are the ones that will shape the reader’s relationship with them. They are also the ones that will show themselves according to their position in this space and the perspectives they adopt, which will depend first on their movement. S.E.N.S’s map is definitely one that “disrupt[s] the sense of coherence and of stability” (Massey: 2005, 109), therefore reflecting the relation of the book’s form with a typical comics layout. The blank map is a map full of potential, mirroring S.E.N.S’s ongoing spatial reconfigurations through the changes of set and therefore calling upon Massey’s reading of the notion of the chance of space, which “lies within the constant formation of spatial configurations, those complex mixtures of planned spatiality and happenstance positionings-in-relation-to-each-other” (116). She adds, “It is in the happenstance juxtaposition in the unforeseen … in the internal irruption, in the impossibility of closure, in the finding of yourself next door to alterity … that the chance of space is to be found.” This reflection has much in common with S.E.N.S’s graphic and narrative structure. It is precisely by not showing a juxtaposition of panels in the page that S.E.N.S’s other formats go on contributing to its own expansion. The element of surprise is present in the turning of the panel/panel, in the room-to-room walking in the gallery, and in the search of routes in the VR game.

The difference between the chance of space that Massey relates to and what Mathieu gives us is that, in Mathieu’s work, this chance is only apparent − it is a hoax. As stated before,S.E.N.S is a unicursal labyrinth, so its multicursality is always apparent. The book is read from beginning to end, the installation is traveled from entrance to exit, and the game, although letting us move, makes us choose only one path to its finish line. There is no chance. There is no analog nor digital apparatus or device that generates random routes or similar tricks: everything is architected for one route and one ending. The one-panel-per-page structure serves this because it does not even give the readers the chance of looking randomly to a grid of panels in one page. The passage from one page to the next, the pause and the surprise it creates, justify Thierry Groensteen’s reference to “a more important gutter [that] represents the passage from one page to the following” (60), and highlights the fact that the choice of the page layout has a narrative and meaningful outcome. The page, in this case a single framed panel, becomes the narrative device that sets the pace of the reading, with the particularity of inducing a slow rhythm that seems to be reinforced by the minimal, but complex aesthetics of each panel, which soaks the reader in.

According to Yi-Fu Tuan (2001), space reflects the possibility of movement, while place indicates a pause (6), and it is precisely a pause in the movement that makes it possible to create a place. In S.E.N.S, there is only one way that a place can be created, and this is through the presence of arrows that pop up along the desert. In such an environment, devoid of significant places, the momentary appearances of the arrows are the sole indicators of the places/pauses, and more importantly, indicators of the intersection of the concepts of space and place, while constituting a route, indicating the movement from one place to another. Besides this, they reveal the pause in the movement, which, however minimal, “makes it possible to locations become places,” as if our stopping marks the spot, imprinting it where was only trace (Tuan 6). Similar to the principle of exposure time in photography, each instant the eye stops, it creates an image, explains Tuan (161), reinforcing, like Noe defends, the act of seeing as an action in itself. However, the pause Tuan refers to is fake, in the sense that it only corresponds to the instant of an action that occurs, like closing and opening the eyelid. Here, one can draw from Martin Thiering’s work on spatial semiotics and refer to them as “course-maintaining devices” that “shape and determine a detailed topographical map” (32). However, in this specific case, the arrows trace a route without leaving a trace. Also, being so alike, in an environment devoid of significant places, they do not contribute to the categorization of the navigational geographies at hand. Yet, they still constitute ghostly landmarks, part of an ever-vanishing route that is only “fixed” − so to speak − as a mental map. In the end, it should be clear that these are the elements that allow a route to be outlined, evidencing not only the space, but also time, since they symbolize a pause that marks the movement between a previous “road” and a new one. Considering this, each arrow points to a goal that is another arrow, thus, the arrow is also a goal in itself. It’s like what the character Allie from Jim Jarmusch’s Permanent Vacation (1980) says: “But what’s a story anyway except one of those connect the dots drawings that in the end forms a picture?” It is arrow by arrow that the route goes on forming in S.E.N.S, “drawing” the narrative.

David Herman, as analyzed by Marie-Laure Ryan, distinguishes between map and course: while one captures a territory away from the person who sees it, the other “temporizes the experience of space by revealing one visual frame at a time” (73).^[11] We can then assume, taking into both Tuan and Herman, that each page turned corresponds to a spatial and temporal advance punctuated by the arrows and that each of them points to − and is − a goal that suggests bodily movement, punctuates space and “anchors time,” to borrow Tuan’s expression (187).^[12]

Which space is this, then, that even without center or periphery continues on being produced by movements? The Deleuzian notion of nomadic space − the desert − serves to emphasize the character’s and the reader’s traces in a route made of constant variations. The minimal set is transversed by a body that traces a route in a desert constantly interrupted by arrows that are revealed only for a brief moment and erased as soon as they fulfil their purpose, as if they are “dislocated from the trajectory” (cf. Deleuze 1003), letting go unnoticed a multiplicity of other places and other routes. That is, every time the character follows one of the arrows, that event/place becomes registered as one of the possibilities of the labyrinth.

The space and the trajectory represented by S.E.N.S apparently doesn’t have a pre-defined limit nor a pre-organized route/itinerary. Whenever the character finds himself in a momentary stop, searching − or waiting − for the arrow, that event/place stays registered as one of the possibilities of the labyrinth, which seems to repeat itself in places and pauses. This constant reconfiguration of space translates the dynamic of the “smooth” space as a space of passage which vastness is susceptible of being “striated by the fall of bodies” (Deleuze 974), also suggesting the intermittent relationship between virtual and actual. Accordingly, in the same way that the desert space, which exists deprived of hierarchies, can be interrupted by a geographic organization and let itself be inhabited by certain elements, so the virtual is always on its way to the actual, or, in Brian Massumi’s words, “reality-given” (147). The reconfiguration of the smooth space is similar to the one in the book and it happens each time the reader provides it with a physical and intellectual use, reflecting a relational event that depends on the dynamics of interpretation. These are dependent on that relational event. The reconfigurations ofS.E.N.S take place by way of subtle changes in setting, which consist in the interference of familiar objects into the desert space. Despite giving us only a glimpse of belonging, these visual inserts cannot properly guide the visitor because they are just mirrors of some reality. However, they simulate the feeling of being situated in a well-known reality, and, in addition, take part in the production of movement in that space, their succession marking the speed and the advance of the narrative. So, it is by facing their presence that the visitor is misled to believe that they will eventually get “somewhere” if they keep moving. In fact, they are always getting somewhere, but never out of there.

Handheld Devices II

One day you are here. You are like Gregor Samsa in Kafka’s story. You wake up and find that you are present.

-Alva Noe, Varieties of Presence (13)

In his interview for MAS Context, Marc-Antoine Mathieu states that he considers himself “more as an architect than as a storyteller,” since he feels that he is “more a space and time manager than a narrator. … It creates shapes/forms but they are shapes that the reader will have to complete. The reader is the one who has to designate them completely. That is the challenge” (“Labyrinths and Metaphysical…”). This idea seems close to the baroque literature’s cult of pleasure and play and its take on the competitive relation between author and reader, built on the basis of a defying author and a reader that looks for an active reading (or, if we prefer, ergodic^[13]). This relationship was mirrored in literature whose design is carefully crafted to turn texts into visual enigmas and poetic labyrinths close to what we now perceive as concrete poetry.^[14] The text is a site for competition, or better still, “virtual competition” (cf. Caillois 32). The author sets up the challenge (or labyrinth) and the reader pursuits it and decipher its enigma in order to − hopefully − win the prize: the completion of the reading, access to the whole of the text. InS.E.N.S, the author is the architect of a desert. Through the use of ludic tools like codes to decipher and by inserting representations of familiar objects throughout the narrative, Mathieu organizes a space that, layer by layer, plants the illusion that character and reader can come out of the maze.Both book and desert are navigated one through the other, through a reading movement proposed by their visual elements. While reading the book you go through the desert, whereas you can only read the book if by traversing the desert. It is this reciprocal action that solidifies the virtual potentiality of the book and of the character. The use of the narrative constituents comes forward in the form of the reader’s and the object’s co-dependency. The world and the individual are not simply there, nor do they simply happen. They activate one another. Vision must be activated by something. As in the example of the photographic exposure given by Tuan, where time and device relate with the object − without which nothing would happen/appear − there is also a need for sight/vision to work in relation with something and form an action. Like the photographic camera has its own way of functioning to simulate the physiological processes of sight, the sight has its internal, physiological functioning that works in order to produce the act of seeing, which needs the conjunction of gears and movements of all the internal pieces of the eye device.

For Noe, presence is something fragile that has to be acquired (2012: 3). It is his belief that one only achieves presence if one has and makes use of the skills to access it. On the one hand, vision has to be activated by an action. On the other, presence is available only through interaction with the world. Seeing is the action that, along with thinking, gives access to presence, thereby activating the object in front of us. Besides this, though the usual handling of the book can seem to relegate our body to a second plane by way of a feeling of immersion, it is also through that same handling that that same body is manifested. In other words, the familiarity with the book format and consecutively, with the movements associated with reading, are the factors that allow us to understand the object and to make it present. When visiting the installation, walking becomes a conscious action that leads the reader through the narrative. As happens when they find the map in the book, going from room to room in the installation puts us in contact with some unexpected off-the-wall elements, making us consciously and physically relate to them. In both cases, we need our bodily movement to get through the object, as well as having our perception activated by the interaction with each environment. It is the actions of seeing and perceiving combined, along with the cognitive processes of thinking, that give access to the “expanded book”: the paper one, that finds ways to spread to the reading space, and the installed one that activates and absorbs us.

In his thesis, Daniel Goodbrey makes argues that architectural mediated comics exhibit a clear tension between the freeform exploration inherent in the three-dimensional space of an installation and the fixed progression dictated by the arrangement of panels in a sequence. Further complicating this relationship, architectural spaces may also impose their own raster of reading on a sequence of panels. (Goodbrey 104)

It is as if the space presents itself as yet another element of the competition, for the author first and then for the reader. The navigation of a space may contradict the usual navigation of a comic, as was the case of the exhibition PoComUk (2003), where the narrative had to be read from right to left because that was what the inherent architectural characteristics of the Institute of Contemporary Art dictated. This is one more example of how a specific space can turn into a reading and narrative device. In the S.E.N.S installation, the big and small rooms, the corridors, and even the terrace conduct the readers in their pursuit of an ending.

Both the installation’s and book’s respective materiality may propose to readers a forgetfulness of themselves, while that same materiality will provide them with a particular understanding and sustain the devices as ‘manifestly present’; in that same paradoxical way, the more that readers move to read, to open maps, and to decode messages, the more they forget their moving bodies. At the same time, there is an anonymous character, unconscious of his own movements, but who is actualized by our vision and our thoughts. Ultimately, this is a reciprocal relationship wherein the character moves because we move, and we move because he moves. This confluence is a way to connect the reader with the character that ends up being the representation of the readers themselves. This is a relevant manifestation of the book, which admits the demonstration of our skills as readers to bring the whole object into focus. Like in baroque literature, the reader should be up to the task of decoding the literary object and message. Or, in the words of Noe:

The world shows up for us in experience only insofar as we know how to contact with it, or, to use a different metaphor, only insofar as we are able to bring it to focus. … We achieve access to the world around us through skilful engagement we acquire and deploy the skills needed to bring the world into focus. (2)

This mutual game of appearing/disappearing echoes an intersection between actualization and virtualization, only made possible because of the immediate experience of the object, that is, of the interaction that is generated in order to activate the reader’s vision. To travel the space of the book implies going beyond its physical limits, although always connected with the physical/actual spaces. This travelling implies that the reader must use their whole body and not only the ocular movement to understand it. This is how the book’s potentialities unfold and unroll and turn it into an agent of an alternate reality, which also exists between “smooth” and “striated.”

How we perceive the object, then, depends on what we do to relate to it (cf. Noe, 2012: 47). By looking at the question posed by Derrida in “On Touching” − “If two gazes look into each other’s eyes, can one then say that they are touching?” (2) − perhaps one can ask, “If my vision activates the object, can I say that I have touched it?” In the same line of thought, Noe states that “You can’t hold hands with a picture; but you can caress it with your eyes” (2012: 104). Thus, to travel the spaces of the book or of the gallery entails using our physical attributes to change those devices in order to make them appear and go beyond their limits. Once this happens, these objects’ potentialities unfold and turn them into agents of the reader’s actualization. And space is a crucial element for those potentialities to unfold because it conveys meaning in itself. It is there where the meaning-making processes and the actualization − the production of presence, if we prefer − of both reader and object take place. Whatever its form, space is a body to relate to and, at the same time, where other spaces/bodies come together and are confronted with each other. Opposing the notion of retaining the spectator to achieve immersion, S.E.N.S is instead about providing them with fluidity.

Noe argues that our perspective is conditioned by our perceptual relation with what we face (58). In this manner, our perception of things depends on our point of view. Therefore, our positioning as viewers will condition our relation with the objects. The changes in perspective originate with and are originated by our movements. This situation reveals a feedback loop between perception and movement: we move because we perceive something that captures our attention and we perceive other phenomena because we move towards it. The movement of our hands can be replaced by the movement of our eyes, bringing one back to Derrida’s aforementioned question. Thus, the interrelation between eye and route reflects the relationship between the eye-focusing mechanism and movement/action.

In terms of perspective options, in the S.E.N.S installation we cannot really choose between perspectives, but are instead able to experiment with both by way of the composition of the work. Looking at the videos in the installation, our position is similar to the third-person perspective, but when walking through actual sand or interacting with some objects (like the chair, the table, and its drawer with the compasses), we have no option but to use our own perspective. As for the book, we tend to have a third-person perspective, yet, when finding the map, a first-person perspective is simulated. I say simulated because it is the reader who opens the paper map, but we have to pay attention to the image of the character’s hands on the left side of the map, like he’s holding the book we’re also holding. It’s like a momentary fusion between character and reader, which highlights the reciprocal relationship between the reader and the character (and the author). They may not be one and the same, but they share spaces. The presentation space, be it book or installation, triggers actions that turn us into the interruption that actualizes the fluid space of the desert. Consequently, the awareness of those actions and movements, along with the self-consciousness of our role in the event taking place, actualizes us as readers. So, rather than being detached from reality, S.E.N.S’s reader is hooked in a route of small pauses that, in their turn, make the reader consider the proximity of the gesture.

Perhaps the reason why every format of S.E.N.S proposes the feeling of moving in an alternate reality where we as readers and visitors, are more aware of our surroundings is because its structure is in itself a holder of a whole conceptual relation: a relation between the object and the reader that goes from its material properties to the profile of the mysterious character. S.E.N.S’s structure supports a continuous exchange of meaning-making processes that end in a codependence and co-actualization relationship between reader and work. More importantly, and related with the next part of the article, is that this structure expands each time it is rewritten by whatever means available, absorbing them also for its interpretative expansion. In this way, the S.E.N.S installation, though not attached to any VR or AR devices, can nonetheless achieve the feeling of augmentation by means of animated screens inside a backlit room that allow the narrative to reveal itself, respecting the book’s intention and pace and adding to it in terms of experience.

Handheld Devices III

“Space is no longer that one the one in Dioptrica, … it is a calculated space from me as a point or zero degree of spatiality. I don’t see according to an exterior envelope, I live it from the inside…. Be it as it is, the world is all around me, not in front of me.

-Maurice Merleau-Ponty, O Olho e o Espírito (48)

Virtual space is dependent on the actual space and on the physicality of the readers’ body and the devices’ hardware to get to both image and imagination. Being lost is often associated with being in the dark. “I’m in the dark,” we often say when we do not understand something or do not know something. In the same way, we are in the dark before passing through the S.E.N.S door described in the first part of this article. When we feel lost, we look for something that is recognizable to serve as a point of reference. In taking the example of someone who gets lost in the dark of a forest, Tuan (2001: 36) says that when our front space is illuminated, it serves as a vehicle for recognizing what surrounds us. The feeling of being lost vanishes the moment we recognize what lies before us, even if we do not know where we are.

Ken Hillis’s book Digital Sensations (1999) and article “Toward the Light ‘within’” (1999) make clear that the evolution of optical technologies, from the discovery of perspective to VR, goes back to the Platonic questioning of the world as an illusion and a constant search for the capture of this illusion of the world. Through techniques and tricks, we have been trying to better understand our experience of the world from new points of view, eventually, from the same points of view captured by new inventions. This is one of the reasons for the importance of the development of perspective in painting during the Renaissance, in conjunction with the development of the panoramic views in both painting and photography around the 18th/19th century. These two technologies are ancestors of what we call VR, in that they aim to keep the viewer/player at the center of a technical construction that must be able to alter our perception so that the feeling of immersion is as great as possible. This is why what differentiates the common use of cyberspace and immersive VR is the relation that the user has with the environment, as Ryan explains:

Cyberspace projects not a continuous territory but a relatively loose net made of links and nodes, of routes and destinations, with nothing in between. The destinations, or sites, may be centers of interest, but the connecting routes are not. Travel from site to site is not a voyage through a developing landscape but an instantaneous jump that negates the body, since material bodies can move through space only by traversing it one point at a time. (3)

It is the fluidity that VR provides that distinguishes it from a regular digital/screen environment. And it was essentially this that led Red Corner, the indie company in charge of creating S.E.N.S VR, to develop the game. This format, they say, is closer to the feeling that they had in reading the book than to developing a game in a 3D environment for regular screen use. This would normalize the game and disregard S.E.N.S from its strong point, which is the idea of fluid passage through space, ever so present in the book’s one-panel page layout and the flowing animations on the screens of the installation.

Here I question whether the use of devices such as the Head-Mounted Display (HMD)^[15], can break the feeling of VR immersion. It is the HMD that allows the recognition of the head movements which, in turn, are responsible for the dynamic (re)construction of the world in which we enter. Therefore, the prevalent idea that VR detaches us from physical spaces or makes us forget the body becomes difficult to accept, due to our position. Our relation with the HMD, although representative of a supposed freedom and unlimited access to the unknown, keeps us in one space − the place where our bodies relate to actions, where they should perform to get the narrative/game going. That is, just as when we browse the book, wandering through the desert, in the VR environment the set also seems to pass us by because, as long as we activate it, it is the animation that moves towards us.

The transposition of S.E.N.S’s book space into the VR environment, which is generally said to be more immersive, allows us to explore using the perception of monocular depth. From the moment the HMD is placed on our head, a new spatial perspective is activated, which brings with it new mobility and/or lighting effects, as well as new organizational and relational relationships between objects. This not only helps regulate the user’s perception (Hillis, Digital Sensations 196), but will also affect their emotional state and visual perception, which in turn affect each other (Trachtman, 2012: 87). In an environment such as VR, we will adjust our visual accommodation because the nature of the space in which we are entering means that we have to adjust our usual use of binocular vision for a monocular vision depth (ibid: 107).

HMD takes advantage of the knowledge of the ocular mechanism to close our eyes to the outside, in a way. This leads us to Hillis’s suggestion that VR enhances our self-enlightenment (“Toward the Light ‘within’” 36), highly linked to a spiritual thought that is exhausted in the search for light and truth, and draws attention to the orthodox idea that it is by the light, and only by it, that the world is shown in our eyes. Hillis transposes theory to the scope of VR and explains:

Immersive VEs combine implicit Platonic and Neoplatonic spatial relationships between seer and light. They first position users “at the margin” of the field of view they illuminate. At this point, eyes glued to the interface, the viewer wearing an HMD replicates the Neoplatonic truth seeker dazzled by looking into the light. However, this viewer is much closer to dazzlement’s source: the HMD, acting as a conduit, replaces the space between the subject and the light. … The Platonic realm of Ideal forms now not only “comes into view” but, by use of the (Neoplatonic) HMD, seems inhabitable as the Ideal made manifest. (Digital Sensations 176)

Despite its relevance, the question of the materialization of the Platonic “Ideal” through the HMD, seems insufficient without referring the actions that the device calls upon, especially when talking about relationships between being/seer, light, and space, where motor action is essential to their manifestation.

The HMD allows visual shifting that gives the impression of movement. It only works if the actual movement of the user indicates it, be it ocular or another kind.S.E.N.S VR does not require the user to move other parts of the body besides the head, even if these can (and likely will) occur. In fact, once the path selection is made by focusing the eye towards the arrow that we are going to follow, the user cannot even turn their head. Whether it is done or not, the mere possibility of being within S.E.N.S puts at the user’s disposal possibilities of perception that are impossible to obtain through the book or the installation, since our perspective is no longer conditioned by what is already inscribed in the paper.

Drawing on from Noe’s notion that our perception of things depends on our point of view, we can say that the position of the user within the game, such as their position as a player, will condition their relationship with the objects. The game was developed taking into account the perspectives of the first and third person, who had already been created by Mathieu for the book. The difference is that in the book, access to these perspectives has a significantly smaller influence on the exploration of the objects that the reader comes across, while the in-game perspective variations allows them to capture differences in their field of vision. When looking at a scenario where the character is included, the field of vision appears to be wider and therefore does not require as much physical movement to look for clues. If, on the other hand, the first-person view is selected, the user will look at things from within the character, and their field of vision seems to have diminished, so they will have to turn their head if they want to see what surrounds them.

These changes of perspective have their origins in the movements we make that consequently change their profile, and they originate movements because, from this perspective, we will select our next action. The naturalness with which we are faced with the changes of scenery is the responsibility of the automatic processes of visual accommodation. Noe puts it this way: “When we approach an object, it appears in our field of vision. When we move around it, its profile changes” (2012: 47). In S.E.N.S VR we can explore in depth the labyrinth we could only follow in the book. In this process, the movement of our hands, which advances the way ahead of me, is replaced by that of the head and the eye, responsible for animating the arrows that show/indicate the way. This interrelation of the eye with the pathway reflects the relationship between the focusing mechanism and movement. The use of HMD, together with a high level of concentration in the ocular movement and a quasi-static position of the whole body (which does not move along with the vision) drives us toward a bodily self-awareness that can be as disruptive to an immersive experience, especially in a VR environment. I would therefore argue that the S.E.N.S installation environment is much more immersive and closer to an augmented experience than the HMD experience.

There are two key points related to the world created by Marc-Antoine Mathieu that demonstrate why this is so. In considering himself to be more of an architect than a narrator, Mathieu unconsciously enriches the S.E.N.S VR experience, since the development of the game does not need to undergo an intrusive adaptation. In addition to this, Mathieu creates a character that, unlike the seemingly most obvious option, is not our avatar, but rather a character with whom we relate in a unique way, not being other nor the same − perhaps a counterpart within us. Consequently, the awareness that we are both inside and outside ofS.E.N.S is what should convey its formats. This is a way of leaving us in a limbo, in the liminal space. A main factor for this is the slowness of the game, contradicting the usual expected dynamic of challenges and counter-clock running. And this is what makes me rethink what Groensteen argues in the fourth chapter of Comics and Narration (2013):

How does reading from a screen differ from reading from the printed page? It entails the loss of a very strong, affectively charged object relation: the physical handling of the book, which involves both arms, or even the entire upper body, is replaced by intermittent pressure on the mouse from the reader’s index finger. There is certainly a loss, in relation to motivity, participation, tactile (and occasionally olfactory) sensations, and even interactivity. (66)

Although understanding where this argument comes from, and although agreeing with it to some extent, I must highlight the contribution of this VR environment to the overall reading experience of the work. S.E.N.S is not only a book, an installation, and/or a VR game; it is an environment in itself. So, when the reader/visitor/player is experiencing the game, they are not just re-reading. It is a new experience altogether of the spatial interruptions, of the objects, of the map, of the arrows. It is not, however, the same kind of pause that Groensteen refers to when talking about zooming tools. Referring to these, he says they “[presuppose] a prolonged pause over an image and so breaks the rhythm of reading. … immersion in the drawing and immersion in the fictional world seem to enter into contradiction” (72).

This is not the case for S.E.N.S First, because there is not exactly a zooming tool working; second, and most importantly, because instead of aleatory pauses related to digital mechanisms and engagement, it pursues an experience of slowness and silence in all its forms. In this case, I would disagree with Groensteen when he declares that, “What disappears along with the self-containment of the work is the spatial memory that was associated with it” (66). At least when it comes to S.E.N.S VR, the goal is to take advantage of its obligatory slowness as a persuasive way to gather attention precisely to a spatial memory. There is no speeding up in this game. We wait until the walking in the sand is done or until the wave passes. We wait, and by waiting, we soak in the memory of its minimalist desert. Besides this, a character in a metamorphic labyrinth, and the inevitability of death seems to serve a clear message to identify with the character. However, there are at least two moments in the narrative that reveal the opposite and that S.E.N.S VR solves. The first is when we find the book inside the book with the map, and it concerns the character’s demand, which is to find the missing page of the book. And the second is when the character is confronted by the mirror, an encounter with self-consciousness − his and ours − and the consciousness of separation − what is in the mirror is what is in the book, which is the “other.”

The automatic manipulation of the book may make it more immersive and help us lose ourselves in a maze and confuse our image with the mirror image. The fact that in S.E.N.S VR there is an awareness that one is outside, and that only part of our body is needed to generate the frontal space, places us in the correct position. And this is the position of the outsider, although one with inside access. The first-person perspective (which actually remains a third person) allows us to understand that we are between spaces and that we are a kind of interruption − an intruder? − that actualizes the fluid space of the desert. Thus, S.E.N.S VR seems to achieve an improbable balance between (little) immersion, body awareness, and narrative.

Nonetheless, we should be aware of a question posed by Noe when recalling that “Traditional approaches to perception have found it impossible to avoid thinking of vision as … a process whereby a model is made in our heads.”: “Why build an internal model when the world can serve as its own model?” It was in building on this notion that AR came into being. Unlike with VR, with AR the screen gains significance by transcending its role as a tool that links spaces, in order to become a vehicle that allows character and reader to connect through different spaces. For example, in Modern Polaxis, the reader will enter the storyworld through the use of a particular device that enables a set of visual and textual layers to connect with the visual and textual discourses already present in the book. As opposed to VR, which simulates a vast space constrained inside a box, AR tries to free us through the combination of physical and actual spaces. This combination allows fictional and actual worlds to collide, too, as is the case of Modern Polaxis. Besides its technological attributes, Modern Polaxis’ main character believes he is a projection from another plane, and his search for the projectionist makes him wander like a traveler. However, contrary to what happens with the character of S.E.N.S, he invokes a great amount of real and imaginary sets (plus the reader’s connection). Admitting that this has much to do with the Platonic and Neoplatonic theories, this work demands further study, which I will leave for another opportunity. As for S.E.N.S, its choice of containing its vastness in a VR game seems to puzzle, and even provoke, us as by expressing the (almost) obvious absurdity of life that is: there is nowhere else to go from here. In the same manner, character and reader are set to go on walking in circles until the ride stops.

Final Considerations

To acknowledge that presence is achieved and that it is achieved in full understanding of its manifest fragility is really to give up the idea that the world shows up as a remote object of contemplation. Perception is a transaction; it is the sharing of a situation with what you perceive.

-Alva Noe, Varieties of Presence (3)

Space is an active participant in the shaping of stories, not only for writers, narrators, and characters, but also for readers. This is why sometimes the materialization of our thoughts and of the images in our minds can feel like a poor resolution, as if a diminishing of our imagination. As I have said, much has changed since McCloud introduced the notion of the “infinite canvas.” At the same time, much remains the same. The truth is, digital comics try to show us one more trick of movement, of color, of touching, of sound. No wonder scholars like Groensteen still consider that “there is no need to bring comics to life, since it is an art form that is already complete” (70) or that “the hypermedium … deprives comics art of its qualities of transparency” (76). I must partially agree with this when considering most webcomics and comics apps. However, I cannot agree that comics should not relate to the digital field at all, because on the one hand, digital tools allow us to explore readership and, on the other hand, being “already complete” may read as “stagnated,” and I think comics are anything but. Comics have huge potential for relating with other media and shaping themselves. Of course, there will always be issues regarding its definition or even its cultural or academic place, but that is not about the purpose of this article. This article concerns comics’ potential to expand through digital media, architectural spaces, or a mixed environment. The “augmented space” I refer to is that of our experience as readers. It has little to do with digital mechanisms. I believe one should focus on what these intermedia and transmedia relations can propose in terms of enhancing their narrative traits. What can the reader gain from these readings?

Wolfgang Iser’s theory of aesthetics of reception tells us that the text waits for the reader to fill in the blanks. If this idea of the reader as an agent that actualizes the book is established, then we need to reflect on how the actualization of the text happens in virtue of the reading objects’ specificities. Because in S.E.N.S each page is a panel, the turning of the pages is equivalent to the margins of a panel, the gutter. Thus, we are facing a type of reading where the filling of the blank spaces, besides only being possible through the physical contact with the book, implies that such contact is significant. By means of a gesture as a producer of meaning, this form of contact contributes to the progression of the narrative. It is, then, through this physical and cognitive relationship between reader and book that both will appear to one another (cf, Alva Noe, 2012: 2). Frame-to-frame, the book provides a feeling of immersion that is fed by an automatic and successive turn of the page, which, if not interrupted, can relegate the body to the background rather than summon it to the “space of our attention”(cf. Noe, 2012: 12). Adding to this are our physical and cognitive contributions to the meaning of what we are experiencing by means of what different media allures to us.

In his interview with MAS Context, Marc-Antoine Mathieu also said of the labyrinth, “[t]he interesting thing about [it] is the experience of losing our references; it means the experience of losing ourselves, the loss of our own reality, or of a thing called reality” (“Labyrinths and Metaphysical…”). S.E.N.S’s labyrinth traces a Borgesian universe and taints it with Camusian absurdity. Instead of the feeling of being lost in reality, it induces a feeling of fragility, in the same sense that Noe gives to the fragility of presence. At first, it is as if it restricts the experience when comparing to a more appealing experience that AR can achieve. However, it is also as if it is a container of its own world. By sharing little space with the actual reality, instead of adapting to the “exterior” environment, is as if it becomes one elliptical mirror that only slowly responds to our image.

Finally, we can say that it is through works like S.E.N.S that the architectural space reveals its own interactive nature, be it through serving as the actual space of both reading and action, the vehicle for spatial representations, or a device from which to look at other spaces. The unfolding of space within discursive spaces contributes to a reading process that reflects a multiplicity of spatial, narrative, and interpretative dynamics, through an ongoing relation between recipient, objects, and reader (in this case, book, architectural, and virtual spaces). As such, the worlds suggested by these interactions, and especially the movements and transgressions that characters and readers alike perform, confirm space as an agent of its own expansion. Not only is the reader lured into the fictional world, but the actual space also becomes fictional through its ‘real features,’ as is the case with the installation. Also in this case, the combination of both fictional and non-fictional elements that may be induced propose a “jointly immersive” environment (cf. Monfort et al.), similar to AR, which leads us to thinking that an Expanded or Extended Space may or may not result from digital devices. A ‘jointly immersive’ environment can suppose the junction of fictional and actual spaces of only a book and the reader if the narrative is designed to command it. The augmented space can be proposed by different means and the augmented reading will have its effect even if only through a book.

Notes

<a name=”footnote-1″></a>[1] Please visit the link to watch the installation’s trailer by LiFE Saint-Nazaire on Vimeo: https://vimeo.com/134090308.

<a name=”footnote-2″></a>[2] Please visit the link to watch the game’s trailer by Red Corner on YouTube: https://www.youtube.com/watch?v=KoJLZ2u_Asg.

<a name=”footnote-3″></a>[3] The Tarquin Engine is a zooming tool that, although useful to navigate along multicursal webcomics, does not feel very organic in the sense that the reader will only zoom in on one panel each time and go back. At the same time, in the kind of work Goodbrey produces and any other multicursal and game-like work, such as Jason Shiga’s Meanwhile app version (2010), it can be fun to use and add a bit of difficulty to the reading by delaying its results. Groensteen would refer to this type of zooming tool as something “rarely used for narrative purposes, such as making a tiny detail visible” (72).

<a name=”footnote-4″></a>[4] To avoid confusion, I will refer to the book as S.E.N.S and to the game as S.E.N.S VR. I will refer to the installation simply as “installation.”

<a name=”footnote-5″></a>[5] Labyrinthium (2013) was inspired by Jorge Luis Borges’ “The circular ruins” and it seems to be a prototype for S.E.N.S, in its turn inspired by Borges’ tale “The two kings and the two labyrinths.”

<a name=”footnote-6″></a>[6] For more on this topic, see my article, “Unfolding and crossing the augmented space: Metaleptic tools and processes within graphic narrative installations,” in Studies in Comics 9.2 (2018).

<a name=”footnote-7″></a>[7] As happens with the aforementioned works by Goodbrey and McKean for Gravett’s “Hypercomics” exhibition and about which you can read in the article mentioned in the previous note.

<a name=”footnote-8″></a>[8] All translations from the Portuguese bibliography are mine.

<a name=”footnote-9″></a>[9] “[R]reading executes the algorithm of meaning encoded in language and typography[.] … [T]his and other templates’ ability to generate a large number of unpredictable associations … offer a model for literature as a complex system and for meaning as an emergent phenomenon” (Portela 39).

<a name=”footnote-10″></a>[10] Curiously, the title S.E.N.S is composed by the initials of “Sens Et Non Sens,” which in a loose translation from the French can be read as: “Meaning and No Meaning,” “Meaning and Nonsense,” and “Direction and No Direction.”

<a name=”footnote-11″></a>[11] In this abstract space, where places are symbols within a continuous space (cf. Relph 26), the speed at which the route is made is marked by a succession of images associated with a greater or lesser advance of the narrative, like defining time through a wider or narrower use of space in comics.

<a name=”footnote-12″></a>[12] “Objects anchor time. They need not … be personal possessions. We can try to reconstruct our past with brief visits to our old neighbourhood and the birthplaces of our parents” (Tuan 187).

<a name=”footnote-13″></a>[13] I here refer to texts whose “intricacies of the medium [are] an integral part of the literary exchange” (Aarseth 1), therefore asking for a nontrivial effort (i. e. beyond the eye movement and the flipping of pages) to be read.

<a name=”footnote-14″></a>[14] For more on this topic, consider consulting Ana Hatherly’s research and artwork (https://gulbenkian.pt/museu/en/ana-hatherly-and-the-baroque-in-garden-made-of-ink) and Dick Higgins’ book, Pattern Poetry: Guide to an Unknown Literature (1987).

<a name=”footnote-15″></a>[15] Although S.E.N.S VR can be played without the HMD, for the purposes of the argument, I will be referring to this situation.

Works Cited