How AR is making video calling more collaborative

At the same time, we’ve become more reliant than ever on video calling to power our social interactions. Early in the pandemic, we at Meta recognized the opportunity to re-imagine video calling and make it better. 

We believe video calling can and should be more personal, more connected, and warmer. One of the ways we’re approaching this is with Group Effects, built on our Spark AR platform and introduced earlier this year. Group Effects are a new type of real-time augmented reality (AR) experience, unique to Meta, that invite people to share moments together through collaborative experiences. With interactive overlays that move across and between screens, and people, Group Effects get us closer to the feeling of shared connection we might actually get in the same room with other people.

Unlike the average AR effect on Instagram that’s largely one-way broadcasting, Group Effects for Instagram and Messenger are a great way to connect or break the ice. They can even be the reason to get on a video call — to play, laugh, or celebrate together and form a new bonding experience for just those people involved.

Today, we’ll highlight launches from the first few months of Group Effects, dive deeper into the opportunity for brands and creators, and offer a peek at what's ahead.

Benefits for brands and creators

Last June, we invited creators and brands to start building Group Effects on Spark AR with our Multipeer API. They responded enthusiastically and have already built an impressive library of AR effects, ranging from lightweight games to interactive experiences to ambient environmental features, such as a crackling fireplace, that help you feel like you’re together with your friends in the same space.

AR effects have been available for Facebook, Instagram and Messenger Stories for some time. Stories can garner millions of impressions for effects creators, but they’re asynchronous and one-way — usually a single person expressing themself or sharing a funny moment, which can limit opportunity for meaningful connection. 

By contrast, Group Effects are intended to be much more intimate — they’re meant for small groups (in fact, optimized for four) —  of friends and family, and inspire engaged, personal connection that can help everyone on the call feel like they’re all in the same space. They offer an entirely new type of experience that people participate in together, in real time. “This is about putting AR in a completely new context, where people interact in a new way,” says Jonathan Sherman, a product manager on the Spark AR team. “Group Effects help overcome the inherent challenges of feeling connected with someone when you’re using video calling. It’s a completely new context for AR that’s real-time, personal, and private.”

And for brands and creators alike, Group Effects represent an exciting, compelling new medium for AR. 

For our brand partners, building Group Effects is an opportunity to develop experiential marketing for the vast number of video calls made every day on Messenger and Instagram. As Locke Dunn, business development manager for AR at Meta, puts it, “Brands always want to be part of the conversation, and here, they can literally be the focus of the conversation in an intimate space where people hang out.”

From turn-based games to transformative worlds 

So, with a new set of tools and creative opportunities, we’ve worked with brands and creators to build examples that illuminate the range of possibilities for Group Effects. Here’s just a sample of some great effects we’ve seen made so far:

Group Effects from brand partners

Heads Up! from Ellen DeGeneres is a new take on her massively successful mobile charades game. Players on a video call try to guess the word that’s on a card on another player’s forehead before they run out of time.

Warp Speed Worm, launching today from AT&T, is a fast-action, multiplayer interactive video calling game for Messenger and Instagram that highlights the enhanced benefits of faster speeds and lower lag times you may observe with AT&T 5G, making your group calls more interactive and helping you better connect with those who matter most. (5G is not required for play).

“Our work with Meta to bring this new AR effect to life shows the power of immersive connection and what’s to come in this space," says Julie Haleluk, Assistant Vice President, 5G Product & Marketing at AT&T. "Video chats can be more than just catching up with your friends onscreen -- this launch demonstrates a video call can be engaging, interactive and fun. Our 5G network is built for this type of entertainment and we’re thrilled to present this experience for all to enjoy."

Effects from creators

Don’t Smile, from Tsarenkova Anastasiya, lets you put funny hats, makeup, or other effects on your friends, while they do the same to you. Neither person sees what the other is doing until a timer goes off — the challenge is to keep from smiling for five seconds.

Boulder Roller, from Eugene Soh, is an adventure game that tasks players with escaping a giant boulder trap.

Draw&Guess Multiplayer, from Dimas Alfaruq, a turn-based game that challenges players to guess a word based on what someone else is drawing — with their face.

Quiz Game, from Jan Trejo, a trivia game in which players answer questions and try to get to 10 points first.

Hot Slug, from Alexey Surnin, Alexander Surning, Sergey Krygin, and Ivan Pavluichenko, is a hot potato-like game where players pass a slug between them and try not to be the one whose head it's on when the timer expires. 

To try Group Effects, head to your Instagram or Messenger app, start a video call or create a Room, tap the smiley face to open the effects tray, and select Group Effects.

What we’ve learned so far

In the months since we launched Group Effects, we’ve had a chance to learn some best practices to pass on to creators and brands so they can build the next wave of great experiences. 

For example, it should be obvious what an effect does, since people will usually discover it when they’re looking through the effects library for something fun to play with on a video call. It must pay off quickly, with instant and sustaining engagement. And it should also have an intuitive language that makes it easy to use without complex instructions.

Being ambient in the background can also be valuable, so a game or intricate effect can be left on without active engagement. “We’ve seen people leave on effects they enjoy for an extended period of time,” explains Kasey Morrison, strategic partner manager for AR partnerships at Meta. “And even if they forget to turn it off, it’s fun to be able to have the effect accidentally triggered by opening your mouth or nodding your head. This provides a spontaneous moment for them to react to, and that can spur more conversation and connection.”

Yet another lesson learned is that effects work best when they’re lightweight and conversational. Anything with a significant barrier to entry is a non-starter. “You usually have just 10 seconds to capture someone’s interest before they try another effect,” Morrison cautions.

Unlocking a new dimension for video calls

More than a century ago, telephone calls changed the world by letting people communicate in real time, regardless of where they were in the world. Then, in 1964, the addition of video made it possible for the first time to see each other’s unspoken facial cues, helping us understand the person on the other end of a call’s feelings or meaning in a way that isn’t conveyed when we can only hear each other. 

But while video calling has improved vastly over the years, and is now an integral part of most people’s lives, its underlying technology — a combination of audio and video channels — hasn’t changed. Now, with AR video calls, we’re adding a new dimension, Group Effects, that will advance communications much as video once did for calls: by helping people feel more connected and together.

There’s never before been a data channel for video calling at the application layer. Group Effects are essentially apps, and because of the Multipeer API, they’re built to form a small communications network between the people using an effect.

Using the Multipeer API, creators build AR effects that communicate with every instance in the same video call. When a participant opens a multipeer effect, an instance of the same effect automatically loads for everyone else on the call.

There’s a few different ways to build Group Effects. Creators can choose from JavaScript or use the Spark AR Studio Patch Editor, a no-code option for those who don’t know JavaScript, or a combination of the two. When someone on a video call interacts with an effect, it impacts the effect on all other participants’ screens as well. Effects can send arbitrary information between callers using the Multipeer API, which allows for effect instances to work in coordination and deliver shared experiences.

Helping people feel together

It’s still early days, but we believe using AR in this context will both help people feel more comfortable turning on their cameras and encourage them to connect in the first place, while making interactive layers a new normal.

At the same time, this kind of communication will be a key use case for future products like AR glasses. Group Effects is a new frontier for AR, taking it beyond asynchronous short-video capture to real-time interactions that enhance the way people connect in a personal way.

As we’ve all seen, video calling has become an integral part of our lives, but it often leaves us wishing for more of the connection we get when we spend quality time with people in the same physical location. We believe that Group Effects can help bridge that gap. “Nothing can replace getting together in person, but if you can’t be together, you need video calling,” Sherman says. “We should help people feel close — and make it as connective an experience as we possibly can.”