Callers can fully embody the characters thanks to real-time skin sampling that matches the 3D character to your unique skin tone before converting it into an AR mask for seamless integration. And this is one of the first times we’ve connected people’s AR experiences across screens in a collaborative, multi-participant environment, where everyone can experience the AR effects simultaneously.
All the AR masks and experiences are powered by Facebook’s own Spark AR, the same technology used by hundreds of thousands of creators around the world. We recently launched a version of this technology so AR creators can also build real-time social video calling experiences of their own. Our Multipeer API includes shared environments or contexts, synchronized effects, and even lightweight AR games.
The ability to embody the characters in AR is an important part of the experience. Similarly, so is honoring the on-stage experience in this digital format. To recreate the live theater experience on screen, we match key lighting cues from the production, relighting each caller in AR as if they were performing on the stage. “Lighting is an important component of visual storytelling in theater — it captures a mood and creates an aesthetic,” explains Art Director John Niehuss. “But it’s particularly important in Harry Potter and the Cursed Child in helping establish specific, iconic locations. Lighting is one of the primary things we could draw on to mimic the play in Curtain Call on Portal.”
All that dynamic lighting presented a unique challenge — you can’t currently add a lot of 3D lights to the Spark AR engine without a significant performance hit. The Creative Engineering team avoided that obstacle by using a shader-based “material capture” or “MatCap” lighting system (also known as a “LitSphere” approach). This let them use more lights than they could have normally. It also allowed those lights to be more carefully art-directed and managed by artists since the inputs were just little images of spheres with light direction painted onto them.
“We wanted to push the limits of what has been done with AR to deliver something truly unique for the passionate fans of everything Harry Potter,” says Niehuss. “And we’re honored to be able to bring some of the magic of Harry Potter and the Cursed Child into people’s homes.”
Spotlight on Story Time
In addition to Curtain Call, we’re adding new multiple choice stories to the Story Time library that will delight audiences of all ages. These new stories let the reader and audience make choices together and explore how those choices impact the story. Build a Story is a classic adventure where you get to choose the hero, setting, conflict, and resolution. Each choice you make defies expectations as you realize that looks can be deceiving. Last but not least, the Extra Ordinary Cafe lets you create your own multi-course meal by selecting the cuisine and each course. Each scene contains unique and absurd food as you open up to trying new things.