Feel the groove: Beat Saber and SUBPAC bring VR rhythm game to hearing impaired

Shedding his headset, he exclaims, “WOW!!! Absolutely amazing!”  Troi’s enthusiasm is typical —Beat Saber sold 1 million copies in its first 10 months and often lands on top-game lists— but his mode of expression is not. He is using British Sign Language.

Troi is among a group of hearing-impaired people who gathered to pilot the game’s new offering for players with hearing loss. The idea originated with the game’s co-developer Jaroslav Beck. A composer himself, Beck had taken to writing music while wearing a SUBPAC, a sporty backpack-looking device that transmits the physical sensation of music through a combination of haptics, proprioception, and bone conduction to allow music producers to better connect to low-frequency sounds. While admiring how the SUBPAC “made you feel the sub-bass in your body as though you were at a large party,” Beck said, had an epiphany: He could pair the SUBPAC with Beat Saber to open the game to people with hearing loss.

He reached out to SUBPAC President and Co-founder John Alexiou, who whole-heartedly agreed. Alexiou and SUBPAC colleague Steve Snooks had been working to extend the music experience to the deaf and hard-of-hearing communities for several years. They were now eager to also help open the worlds of VR and rhythmic gaming. “Music and gaming are unifying cultures that can bring people from around the world together,” Alexiou adds.

For the last 15 years, rhythmic video games have acted as a digital form of physical or occupational therapy. Shortly after Guitar Hero’s launch, stroke patients began playing the game to exercise fine motor skills in their hands and a few medical device companies followed suit. While Beat Saber doesn't currently offer therapeutic benefits, its VR platform can be revelatory for people with hearing challenges. “It feels like being in a different world,” one of the players says. “Like a fantasy world.”

Meanwhile, Beck underwent some revelations of his own. The founder/musician was testing out Beat Saber’s library to see which songs generated strong pulses through the SUBPAC. The experience compelled him to compose music solely using SUBPAC’s technology.  “It totally changes the way you create music when you ‘feel it’ rather than ‘hear it',” says Beck.

Music is not an experience exclusive to those who can hear it. Just ask Chris Fonseca, a dancer and choreographer who also happens to be deaf. “I cannot go a day without listening to music,” he says. “As soon as music plays, the sound and beat spread rhythm across the room and the rhythm slowly travels through my body making me feel the beat, as though I am part of the music.” Part of Alexiou’s mission has been to share this level of joy with those who have little or no relationship to sound.

Whether or not the Beat Saber/SUBPAC combo will catch on rests entirely in the hands of the hearing-impaired community. As a rule, when players feel they have the potential to master a game they will return to it repeatedly. That’s partly what helped with the high retention rate of the original version of Beat Saber. It’s also why Beck repeatedly asks deaf or hearing-impaired players whether they think they will improve. “Definitely,” answers Fonseca. “The more I play, the more I start to understand the rhythms. It’s like learning choreography and gradually picking up the timing of when to hit the cubes and when to move."

Eventually, Fonseca may have a chance to prove his Beat Saber acumen. Beck and his colleagues at Beat Saber are considering a tournament that includes hearing, deaf, and hearing-impaired competitors. “Our goal would be to erase boundaries and let all players play without any advantages or disadvantages on future Beat Saber tournaments,” he says. “There is no reason why somebody who is deaf and feels the music can’t beat somebody who can hear the music.” To which, Fonseco gamely responds, “Oh! It’s on!”