The Obstacles to Animal Motion Capture in Video Game Development

In our never-ending quest for more realism in video games, motion-capture technology has long been employed to create fluid, accurate animations to bring our favorite characters to life. But it's not just humans who are employed in mo-cap, we also use our four-legged friends to create both domestic companions and wildlife in video game development.

Marvel Games and Eidos-Montreal recently teamed up with Twitter account WeRateDogs to show off two motion-capture dogs who suited up for the upcoming Guardians of the Galaxy game. Diego, a golden retriever, helped create animations for Cosmo the telepathic space dog, while his little terrier friend, Gretel, lent her acting expertise to Cosmo's litter of space pups. The dogs were put through their paces, asked to do behaviors like barking, jumping, crawling, digging, spinning, and playing fetch.

This is Diego. He plays Cosmo the telepathic Space Dog in the pupcoming @Marvel Guardians of the Galaxy video game. He was fitted with a high-tech motion capture suit to create all the in-game footage. 14/10 would be an honor to pet #ad pic.twitter.com/xuPGmfn89P

— WeRateDogs® (@dog_rates) October 7, 2021

Fashion Faux-Pas

It looks like a lot of fun and games for the dogs, but animal mo-cap presents unique challenges for animators and developers. For starters, human actors will usually wear spandex suits with reflective markers on their joints and other body landmarks. But we've had to come up with creative methods to dress dogs for the part.

Zach Volker, the lead animator for Call of Duty: Ghosts, worked with two German shepherds, Colin and Ruger, and a Belgian malinois, Rico, to create Riley, a playable military working dog. They started with a suit for dogs with skin conditions, but Volker said that Colin, "got hot very quickly so we could only work in 20-minute increments." Mo-cap supervisor Kristina Adelmeyer added that the dogs originally wore special rubber booties with markers glued on. Their movements proved stiff and unnatural, or, as Volker put it, Colin was "walking around like he's on molasses." To remedy this, they removed the booties altogether and used unobtrusive reflective tape on key points instead of a full-body suit.

Vida Starcevic, community manager for Remedy Entertainment, revealed that in mo-cap experiments with a Staffordshire bull terrier, Uuno, they first tried using a surgical recovery suit for dogs. Starcevic reported that "the thickness of the fabric made him [Uuno] feel uncomfortable and weird." They also resorted to using tape that would come off easily to make Uuno more relaxed.

Apparel aside, being able to put markers in the right places to form a more complete picture is another difficulty when recording animal mo-cap. Digic Pictures, a Hungarian studio that's done cinematic work for big franchises such as Assassin’s Creed, Final Fantasy, and Warhammer, recorded a video of cats swatting at toys, spinning for a tasty treat, and jumping hurdles. You'll notice the cats don't have any markers on their tails. Presumably, this is because cats use their tails for balance and communication, and placing markers there would've made them scared, uncomfortable, or move unnaturally.

Contextual Challenges

Even after giving the booties the boot and scrapping the suit, you're still left with an animal who has no idea what you're doing or why you're doing it. Ruger's trainer, Chris Connell, who worked with the shepherd to provide Riley's movements, described this complication: "It was like, 'Okay dog, pretend that this is a rural environment or in the desert area, and act accordingly,' and Ruger is like, 'Dude, I'm still in the same studio and I don't understand the context of what's going to be different.'" To Ruger and his colleagues, they're not yanking a helicopter pilot from his seat, they're just tearing into the well-padded human biting bag like they've been trained to do.

In March 2021, Expansive Worlds introduced the Bloodhound DLC to theHunter: Call of the Wild. The DLC gives you a canine companion to assist you on your hunting expeditions. In an interview, game designer Arshak Ardeshir discussed the challenges of working with canines on set: "... Unlike with human actors, you can’t tell a dog to 'do that one more time, but stop on your left foot first please.' This made it difficult to predict how the moment-to-moment actions would go. Quite often though, we ended up getting things we needed in places where we didn't expect, and sometimes things that would seem simple just weren’t happening."

Notice that in each of these motion-capture sessions, developers tended to use the same or a similar breed of dog as the one they were trying to portray. It might not be necessary, but it's probably a lot easier to get attack sequences from a German shepherd or Belgian malinois, which are often trained to do police and military work. Similarly, the dogs employed in theHunter: Call of the Wild appear to be a breed of pointer and some type of scent hound, which would have natural inclinations to engage in roaming, tracking, and retrieving behaviors.

Ardeshir said another issue was "creating the dog-to-player interactions," which arose from, "technical hurdles regarding aligning and syncing up multiple characters." This highlights the importance of being able to capture animal interactions with humans, rather than recording the data separately and trying to adjust after the fact. 

Simon James, head of the motion capture department at Traveller's Tales, echoes this sentiment regarding capturing a horse and rider together: "... It's got a different weight to it with a rider on. So as the rider performs combat moves, just as the horse moves around, it has to shift its weight and balance quite differently." The bounces, hand and leg cues, and other subtle movements wouldn't align if the rider and horse were captured in different sessions.

Scarcity Struggles

Where horses are involved, there's another complication when using motion capture for video-game animation. They're a lot larger than dogs and cats, it's harder to find one with a good temperament for the job, they require more complicated setups to record, and you need skilled riders and handlers to prevent injury. The development team for The Last of Us Part II originally wanted to rent out an equestrian center for their horse motion capture but decided to use a studio at Sony instead. Jeremy Yates, the lead animator for the game, joked in a tweet that "pretty much the ENTIRE current world’s supply of rubber matts [sic] were ordered for the floor."

While Sony might have a studio barely large enough to record a horse jumping, if you don't have the funds to perform your own mo-cap project, you have to rely on data collected from independent studios. There's no guarantee they'll have what you need. Simon James noted, "There's very little stunt work available for horses… falling down, rearing up, rider attacks." Indeed, Neil Druckmann, co-president of Naughty Dog, the developer behind The Last of Us Part II, admitted that Naughty Dog shared their horse mo-cap data with Sucker Punch Productions to create horse animation for Ghost of Tsushima. A job shared is a job halved.

It seems like there's no shortage of complications in animal motion capture. But as video games become increasingly grandiose and developers strive for more accuracy, it's clear that the need for the technology isn't going anywhere. Luckily, developers, animators, and animal handlers are constantly coming up with clever solutions to streamline the process and get the data needed. Let's just hope our quadrupedal colleagues are always satisfied with their hourly rate of treats and back scratches.