MAMMA-Mia! Few smartphone cameras is all it takes to create dancing avatars

Tübingen, Germany – Skin-tight motion capture suits covered in markers have seen better days. Their fate might soon be to gather dust on a shelf.

For decades, the gold standard for capturing human movement and transferring this data into the digital world has required actors to wear suits with physical markers that cameras track in a specialized studio. While accurate, this capturing process is time-consuming and expensive. Markers need to be placed manually, they can fall off, and if two people get too close – such as during a hug or a dance – the system fails to track all markers as one person blocks the other from the camera’s view and accuracy drops significantly. Extensive post-processing is needed to recover the motion in such challenging sequences. The creators of the Hollywood blockbusters “Avatar” or “The Lord of the Rings” will know what we mean.

Commercial marker-less systems do exist however they remain expensive and are generally considered to be less accurate than marker-based systems. This problem triggered researchers at the Max Planck Institute for Intelligent Systems (MPI-IS) and at Carnegie Mellon University to develop a computer vision technology that achieves an accuracy comparable to commercial marker-based systems without the tedious manual clean up, and – going one step further – solving the challenging problem of close interaction between people.

Markerless Accurate Multi-person Motion Acquisition, or MAMMA, takes multi-camera video, estimates virtual markers in all camera views, and then fits a human body model (SMPL-X) to these markers. In this way, the network can "see" 512 virtual tracking points directly on people’s skin from regular video frames. Additionally, it can predict poses it has never seen during training. So unlike previous attempts at marker-less capture, which often fail to accurately recover all body shapes and are not publicly available, MAMMA is specifically designed to work perfectly using standard smartphone cameras, and handle complex social interactions and physical contact. Martial art avatars fighting or two digital people dancing Salsa are just as realistic as the alive-and-kicking sportsmen or dancers.

“The difference in pose accuracy between our technology and suit-and-marker based systems was less than one millimeter, yet MAMMA can process the data nearly three times faster than the manual workflow required for traditional systems. It can even work with consumer-grade equipment, like a set of iPhones,” says Hanz Cuevas Velasquez. He is a research scientist in the Perceiving Systems Department at MPI-IS and first author of the MAMMA paper.

Transforming Industries

The researchers anticipate that this technology will have a profound impact across industries such as medicine and biomechanics, virtual reality and the metaverse, sport science, the film or gaming industry, to name a few.

Anastasios Yiannakidis is a research engineer in the same department, and continues: “The system can be used to track patient movement for physical therapy or biomechanical research. Because it requires no markers, it is much less intrusive for patients and can be used in more natural environments rather than a clinical lab.”

By accurately reconstructing both the shape and pose of a person’s body and hands, MAMMA can create lifelike digital avatars for social VR settings, making virtual interactions feel more human and expressive.

Sports coaches could use this technology to analyze the form and technique of multiple athletes simultaneously during practice sessions without disrupting the athletes' movements with bulky gear.

“MAMMA is a game changer in many ways. It allows game developers to capture realistic, contact-rich animations like wrestling or dancing without the overhead of a massive motion capture studio. Our technology will allow smaller, independent studios and research teams to create character movements that look as polished as those in big-budget blockbuster movies,” says Data Team Lead Tsvetelina Alexiadis, another one of the paper authors.

By releasing their data and tools to the academic community, the team of scientists hope to spark a new wave of innovation in how the motion of several people is recorded, to better replicate and hence understand human movement.

About the Research

The MAMMA project is a collaboration between the Max Planck Institute for Intelligent Systems in Germany and Carnegie Mellon University in the USA. The research was presented at the CVPR 2026 conference:  https://mamma.is.tue.mpg.de/ 

Images can be found here.