During March and April 2020, as countries around the world were locking down in response to the coronavirus pandemic, the team accelerated work on Rooms to complete the public launch well ahead of the original plan. Working from their own dining tables and home offices, these engineers accelerated work on a video chats project that had begun in Facebook’s Seattle offices in 2018.
Building Rooms during a global pandemic
“COVID-19 brought huge demand in the market for video calling, but we didn’t yet have a suitable product for consumers,” says Seattle-based software engineer Stephen Shi. He and the other engineers felt the urgency to deliver the feature across iOS, desktop, mobile web, and Messenger Desktop — all while creating an experience that is easily accessible from within Facebook. “Building an entire project in that time frame, at the scale and quality that Facebook needs, is a major engineering achievement,” says Shi.
The team went from public testing to launch on a consolidated time line, launching in the United States on May 14, 2020. “The product also needed leverage within the Facebook platform, giving it wider reach and integration across the family of apps: Facebook, Messenger, Instagram, and WhatsApp,” adds Shi. “Building those access points in parallel involved huge technical challenges as well.”
Real-time video chat is not a new feature for Facebook’s apps. There are more than 150 million video calls on Messenger every day. That’s why Facebook engineers in Seattle were already working on ways to expand video chat availability well before the pandemic. In 2018, they began work on a Messenger video calling project to make it easier to join video chats, even for people who don’t have a Facebook account. “We are always trying to increase the reach and value of video chat,” says Vivek Garg, a Seattle-based engineering manager who’s been working on remote presence for several years. “We’ve added a secure URL format, a safe and secure model for Rooms, and created a version to work on mobile browsers.” The first public test, in January 2019, allowed people to create a link from an ongoing group video chat.
At the outset of the pandemic, a small team of engineers had just pivoted to work on group video calling that allowed people to join a chat whether or not they had a Facebook login. COVID-19 caused the team to step up a few gears. In March 2020, video calling on Messenger and WhatsApp more than doubled in many countries, and views of Facebook Live and Instagram Live videos increased significantly.
Garg says his colleagues quickly began working at warp speed. “People just jumped into this high-priority work,” he explains. “The whole company was able to pivot to this without debating. This is what Facebook is good at — changing gears to ship something. We have a saying, ‘Move fast with stable infra,’ and our engineering systems lets us do this at Facebook scale and quality.” Some engineers initially found it difficult to adjust to working from home, but it wasn’t long before they were collaborating efficiently. “When people joined the project from various places, no one really could tell the difference between their location or team. Boundaries disappeared,” says Garg.
Turning challenges into triumphs
This was about more than an all-hands-on-deck approach to support short-term video call volume: The engineers were creating a full-fledged real-time video feature that could stand up over the long term, explains Shi. “Video calling in the consumer world normally involves around eight people max [in a room] – when you talk to your friends and family,” he says. “But as workloads moved online, we needed a solution that would allow us to support a 50-person call, in a short amount of time.”
One major challenge was building Rooms for people using Facebook, Messenger, WhatsApp, Instagram, and Portal. “We had to build so it doesn’t matter which platform you are coming from, and so you can join a room even if you don’t have a Facebook account,” says Shi.
Engineers were also mindful that Facebook is a platform that serves people all over the world and hence faces huge variations in device quality and network conditions. “We needed to optimize for the experience, whereby people with better networks and devices get high video quality while those with inferior networks and devices can still have a feasible experience,” says Shi.
To optimize Messenger Rooms for the best possible experience for everyone, the engineers also did product testing. This included testing in various countries ahead of the product’s global launch. “Locally, we have network condition simulators that simulate conditions from lots of places — this includes a broad range of test devices, to give us a simulation of the lower-end devices,” explains Shi.
Crunch time
With less than two weeks remaining before the intended launch date, the Seattle team needed to climb a mountain. In testing, the connection rate was not yet high enough for a wider launch. They knew that only certain data center regions could support large conference calls with scores of participants. This required them to tune media quality and capacity across the globe. “The quality for everyone on the call suffers when someone joins via a poor network connection or from a browser, without using the native app,” explains Shi.
To solve for this, engineers varied the rollout and maximum audience size of Messenger Rooms depending on the associated data center region. In some cases, large calls required a trade-off. For example, if a participant’s bandwidth couldn’t support optimal video and audio, then the system would need to sacrifice video quality to maintain a good audio connection, which is essential to understand a conversation. “There are a broad range of conditions that come into play to create one joint experience that’s solid,” says Shi.
Garg explains that engineers also spun up an API surface to allow for smoother integration between the Facebook apps. “It could be called from different apps to create and distribute Rooms,” he says. “When someone clicked to join, we redirected them to the Messenger app if it was installed, or used a mobile browser to host it.”
As part of this work, “we did a lot of engineering work to make sure Messenger Rooms had dynamic awareness,” says Shi. “For instance, you can see an eight-person grid on mobile, a 16-person grid on the web, and you can see the maximum room size changing according to the audience size.” This dynamic awareness was an involved feature to build and craft, particularly in just four days.
An important area of focus was the privacy and security of Messenger Rooms. “We did a lot of work behind the scenes to ensure that people felt safe in the room,” says Shi. This included making sure that links to Rooms are random, so they are not guessable. “Then there is our unique audience model of who is allowed, who needs permission, and which Rooms are viewable, loadable, and joinable,” says Garg.
But at the end of the day, Messenger Rooms is all about connecting and having fun with friends and family. So engineers built fun, interactive features galore. Whether people create a room from Instagram Direct, Messenger, or Facebook, they can host their celebrations; gather a book club; hang out while watching videos, shows, and movies with Watch Together; or just enjoy the AI-powered augmented reality effects, 360-degree backgrounds, and interactive games from the comfort of their couches.
