Yaser Sheikh, the Director of Research at Facebook Reality Labs in Pittsburgh, is deeply invested in creating new and better ways for people to connect, even when they’re on opposite sides of the world. “Most of us, myself included, don’t live in the places where we grew up,” he says. “I’ve spent my life moving from city to city, and each time, I’ve left relationships that are important to me.”
That focus on connection is what’s driving Sheikh’s work leading a project called Codec Avatars, which seeks to overcome the challenges of physical distance between people, and between people and opportunity. Using groundbreaking 3D capture technology and AI systems, Codec Avatars could let people in the future create lifelike virtual avatars of themselves quickly and easily, helping social connections in virtual reality become as natural and common as those in the real world. While avatars have been a staple of video games and apps for years, Sheikh believes incredibly accurate virtual representations of people — those that can perfectly capture a wry smile or a furrowed brow — will be a game changer.
This is the first in a series of blog posts exploring the work happening at Facebook Reality Labs. We’ll take you inside FRL and introduce you to the people helping build the future of connection. Learn more below, and click here to read a note from FRL’s Chief Scientist, Michael Abrash.
Codec Avatars is an active research project today, but it could radically change the way we connect through VR headsets and AR glasses tomorrow. It’s not just about cutting-edge graphics or advanced motion tracking. It’s about making it as natural and effortless to interact with people in virtual reality as it is with someone right in front of you. The challenge lies in creating authentic interactions in artificial environments.
If telepresence lets you feel like you’re somewhere else, then social presence lets you share that sensation with other people. Sheikh talks about two simple but important ways to measure success. “We colloquially refer to this as passing the ‘ego test’ and the ‘mother test,’” he says. “You have to love your avatar and your mother has to love your avatar before the two of you feel comfortable interacting like you would in real life. That’s a really high bar.”
The first time you answered a video call, no one had to tell you why the technology mattered. It brought you closer to everyone — and it meant you could work in your pajamas. The jump from video calls to avatar calls will deliver genuine social presence, a bit like talking to someone in a Star Trek holodeck, where participants could hang out in simulated environments like they were actually there. Getting Codec Avatars to work in a way that’s authentic and comfortable is a huge design challenge the Pittsburgh team has been working on for years, and the team is deeply invested in getting it right.