Parabolic MoCap

2024.6 - 2024.7

Markerless Motion Capture in Parabolic Environment

Motion Capture, Captury

During my internship, we got a chance to work with a singer to make 3D scanning of her and record her movement in a parabolic flight to create virtual scene in Unreal Engine. We researched and designed an innovative workflow that used 11 action cameras instead of usual motion capture cameras, configured the camera rig to ensure stable recording in the plane with no power supply and significant vibration, and optimized the calibration and tracking workflow to be able to take full advantage of limited space in the plane, and the short time (about 8 minutes) available to setup all the cameras and conduct calibration.

My Work

I was in charge of the general research of setting up markerless motion capture system in a parabolic flight, including hardware selection and configuration, camera calibration workflow research and optimization, as well as the data processing after the flight.

Using Action Camera as Motion Capture Cameras

Instead of traditional motion capture cameras, 11 action cameras are used due to their compact size and ability to handle extreme environments. Rigorous testing was conducted in the studio to ensure the camera rigs could handle vibration and record uninterrupted in flight.

Camera Rig

Camera placements and record settings are designed and tested specifically for the limited and challenging space of the zero gravity flight. The camera rigs are designed to be fool-proof, with a preconfigured and coded mounting setup that allowed us to set up all 11 cameras in a less than 10-minute window and perform calibration.

The Flight

Footage Processing & Motion Capturing

The footages were post-processed after the flight before being imported into Captury Live for calibration and tracking. Intrinsic calibrations were manually added and edited afterward to save time during flight and ensure consistency. Floor calibration can also be manually adjusted when the automatic detection result is poor, although it was not needed in this project.

In the end, a calibration error of about 5mm was achieved despite the challenging environment, which helped us to successfully capture the motion in about 70-80% of the footage with the actor present.

Documentation

The experience of this project was documented to provide guidance and insights for conducting non-realtime markerless motion capture using non-standard cameras, especially in complex environments.

Enlin Li

Kanagawa, Japan