Yorick's skull wearing the HoloLens glasses

Theater in augmented reality: Technology bridges audience and stage

Elizabeth Hunter’s productions use video games and augmented reality headsets to place audience members in the perspective of protagonists.

Elizabeth Hunter, assistant professor of drama, envisions a world where you can pull up the latest staging of a classic play on an augmented reality headset in your own living room. Hunter uses Microsoft’s HoloLens, an augmented reality device typically used for research applications, to stage productions that allow the audience to see events from the perspectives of a play’s characters.

“In theater, we always use new and interesting technologies to make art,” Hunter said. “In my work, I think about how meaning gets made with these different tools, especially when the tools involve the audience participating in the product.”

Hunter joined the Washington University faculty in 2021 as part of the first group of hires in the Digital Transformation Initiative, an effort launched by Dean Feng Sheng Hu to bolster and support data scientists and digital scholars across disciplines in Arts & Sciences.

Hunter has a long history with immersive theater that stretches back to her time as the founding director of Sloss Performing Arts/MUSE OF FIRE, an immersive Shakespeare project housed in a restored blast furnace and National Historic Landmark in Birmingham, Alabama. She managed the company of actors and directed their productions, which took audiences around the 30-acre site as actors gave interactive performances.

Elizabeth Hunter strikes a Hamlet pose with a prop skull. 

Despite making digital products, Hunter considers herself to be first a critical theorist. She uses technology to explore the relationship between production and audience. According to Hunter, what’s new about her work now is less the use of augmented reality tech itself and more the theoretical questions it enables her to ask about drama.

Hunter refers to theater itself as a kind of virtual reality where you go to a space in order to experience something that overwrites your daily experience. The HoloLens simply provides an extension of the kind of experience that theater already invites its audiences to explore.

Theaters were not historically the dark, hushed spaces that they are today. Ancient playwrights often wrote the audience into the architecture of their plays. When Henry turns around and speaks to his army in Shakespeare’s Henry V, Hunter notes, he turns to the audience, many of whom would have been soldiers at one point in their lives. Hunter uses technology like the HoloLens or video games to enable audiences to reenact that older mode of spectatorship.

“I started by asking,” Hunter said, “what production choices can I build in that the audience members feel like they’re really part of the story?”

Macbeth's stage directions call for copious amounts of blood, which can be costly for live performances. With a video game, Hunter noted, one can always add more blood.

Hunter’s first experiment involved a video game adaptation of Macbeth called Something Wicked, a side-scroller with simple controls that invited players to understand something about the way Shakespeare’s play works by learning how the game works.

“Gamers can feel how we built game mechanics, and they're like, ‘Oh, wait a minute. Is this a play about backstabbing? I knew it, because I killed him faster by stabbing him in the back.’ You have to play that game by the rules of the world of Macbeth.”

Hunter calls her work “critical making,” using a phrase popularized by information scientist Matt Ratto to describe a process of making a product in conversation with relevant critical literature so that the literature informs the design choices. Making the thing is not the end goal. Instead, she wants to use the making process to create new knowledge about the plays and their critical reception.

Turning to the HoloLens, Hunter flips the idea of audience participation on its head. Instead of the audience being built into the architecture of the play, she wants to place the play inside the architecture of everyday spaces. Her first production using the HoloLens was Bitter Wind: A Greek Tragedy in Mixed Reality, an adaptation of Aeschylus’s Agamemnon.

A screenshot from Bitter Wind

Bitter Wind begins with a ghostly hologram that serves as a guide throughout the play. The HoloLens allows Hunter to project the specter into the real environment that the audience member currently occupies.

“I’m enchanted by this idea of theatricalizing non-theatrical spaces,” Hunter explained. “I’m fascinated by what happens when someone is watching Bitter Wind in their living room and then they turn it off. People remember where the sad girl hologram was standing even in their own space.”

By placing a play like Agamemnon in someone’s living room, Hunter closes the distance between the audience and well-known stories. The actors are not on a raised stage in a dark room, far away from the quiet, orderly people in the auditorium. They are in your living room.

“That personalizing of stories that have seemed separate from us creates an intimacy with those stories that spark curiosity.”