Step into Frame: The Frontier of Immersive Cinema

The film, which blends footage from before and after the blaze in the same locations, captures the majestic space so well that viewers can practically smell the incense mingled with smoke from the fire. “With 13 million visitors each year, Notre Dame is a topic that speaks to everyone,”  Rochereuil says. “It’s also a unique building that involves many communities and conveys strong emotions for believers and nonbelievers.

Virtual reality was the most powerful tool to re-create these feelings, the ‘wow’ effect everyone had when they entered the cathedral. Being able to experience a mass in Notre Dame again, watching the details of the bells, the rosaces [the glorious stained-glass rose windows], and contemplating the beauty of the building … these are things only VR could deliver.”

Virtual reality films are increasingly becoming a sophisticated tool for storytelling, one that guides viewers through the narrative in a novel way and sometimes even lets them participate. In fact, VR cinema changes the relationship between filmmakers and the audience. In VR, filmmakers hand a level of control over to the audience (because you can focus on whatever you want in VR, pausing to study a stained-glass window, for example) which is the exact opposite dynamic seen at the theater or on television. The result is that “wow” effect Rochereuil refers to, which perfectly describes what viewers of immersive cinema experience when, sometimes literally, diving into a film.

Maybe it’s the awe you feel when ushered into the hush of a regal theater for a live performance of The Tempest. The adrenaline rush when you scramble to get away from the fiendish Wolves in the Walls. Or maybe it’s the sudden―and overwhelming― discovery that you’re standing in a burned-out pile of rubble in Kabul. Immersive cinema does not simply paint a picture on the screen for passive enjoyment. It imbues empathy and feeling into the very moment; a sense that you’re not just a casual observer—you’ve been flung into a space where all that exists is the narrative that surrounds you. 

“The sense of presence, for me, is the greatest benefit [of VR cinema],” says Ricardo Laganaro, a Brazilian filmmaker and Chief Storytelling Officer at ARVORE, a VR studio located in Los Angeles and São Paulo. “Immersion, by itself, does not necessarily guarantee the sense of presence.” Presence, he says, only happens when the user’s entire body is part of the experience. “So, this rediscovery of the body at the center of intellectual experience (as we used to do playing during childhood) is the greatest benefit of VR in my view,” he says. “When we remember that we can think with our entire bodies, we can use it to tell and hear stories, and that makes us more human.”

How to tell a story virtually

Early on, though, Oculus producers weren’t exactly sure what to do with the medium.  “When I first started in the Oculus Story Studio, we asked if you could even tell a story in VR,” says Yelena Rachitsky, Executive Producer of Media, Facebook Reality Labs. It was viewed initially as a mode for gaming, so there was no precedent for telling stories in VR. 

Oculus went into full exploration mode. They played with gaming techniques, making some films, like Lost, exploratory and interactive, and others, like Dear Angelica, purely cinematic. And early on, they dipped their creative toes into social justice with Clouds over Sidra (Chris Milk and Gabo Arora), a short 360 VR film about a Syrian refugee.

Oculus picked up its first Emmy with Henry, which won for Outstanding Innovation in Interactive Media. “We never anticipated that one of our first projects would be given such a distinction, and this recognition is not only a testament to our team’s creative and technical achievements but also a validation for the VR storytelling community as a whole,” Henry’s director, Ramiro Lopez Dou, said at the time. “While Henry is just one step in the long journey ahead, we hope this moment inspires storytellers to bring their ideas to this new medium and help shape the future of VR storytelling.” 

Today, Oculus VR experiences range from social justice documentaries to animation and sweeping narratives to newly conceived interactive narrative experiences in which viewers are treated to live-action events, like The Tempest. Oculus is now producing and partnering with dozens of films a year (including social justice films for VR for Good), hosting film festivals, like Tribeca Film Festival Cinema360, and even encouraging well-known traditional directors like Roger Ross Williams, Jon Favreau, and Darren Aronofsky to experiment with VR.

Capturing the moment

Laganaro filmed his short documentary Step to the Line inside a prison, an experience that was emotional for the crew, who had never been inside prison walls — and perfect for VR. He says that Solano Prison, where he filmed much of his footage, was nothing like what he was expecting from all the movies he’d watched featuring prison life. “It is an extremely artificial and tense type of peace,” he says of the calm he first experienced when he entered the main quarters. 

“However, inside the gym where we recorded [the protagonist’s] work with the inmates, it was another world. A place full of life, with people looking into each other’s eyes, close to each other, interacting without filters. Laughing and crying with each other. I understood then that VR, because of the sense of presence, could show in a more legitimate way what real life is like in a prison.”

His biggest challenge? “From the technology to the language of the medium, everything is developing and evolving so fast that the writing process has to be more integrated with the entire production process, not only during pre-production,” he says. “At ARVORE, we created a framework that mixes filmmaking and game development, transforming the screenwriter into a narrative designer that is part of the production team throughout the entire process. It is easier to understand how it works for animation, but it can also work very well in live action if you find ways to prototype your piece a few times (360-degree animated storyboards, recorded rehearsal sessions, etc.) before the actual shooting days.”

New technologies will continue to make VR films exciting and interesting and bring it to the mainstream. But it is the human draw of old-fashioned storytelling that will make immersive cinematic experiences mainstream. “I think that the tools need to be easier for people to make stuff within the storytelling space for a broader range of creators to come in,” Rachitsky says. “Audiences have an expectation of quality. If they go through the effort of putting on a headset, it needs to be worth it.”

The act of “wearing” a screen brings its own set of benefits and perspectives. “In a VR headset, we have a captive audience,” explains filmmaker Celine Tricart, who created The Key. “We control everything the participants see and hear; we are not limited by the edges of a frame anymore. It’s an exhilarating power for storytellers. I’m always amazed by the fact that VR is used to treat phantom pain in amputees, as well as phobias and PTSD. I think we have only scratched the surface of what we can achieve with VR.”

Of course, the flip side is true as well — directors have to let go of their control a bit to allow their viewers to wander and explore, and even participate at times, within the VR realm. Unlike with traditional movies, where your perspective is locked to whatever is on the screen, here you can turn your focus to whatever you want. “When you give your audience the ability to interact, they have a responsibility,” Rachitsky says. The higher stakes can make them feel more engaged and invested in the film. “This is the interactivity of storytelling.”

Recent innovations like hand tracking have expanded interactivity and audience participation even further. The Line, an interactive tale about star crossed puppets set within a scale model of 1940s São Paulo, invites audiences to go hands-on with the narrative — literally. Directed by Laganaro, and Winner of the 2020 Primetime Emmy® Juried Award for Outstanding Innovation in Interactive Media, The Line is at the cutting edge of immersive storytelling.  

“VR will bring the body back to teaching, storytelling, and working,” says Laganaro. “I think we’ll be more human, and better because of it.”