Single Character Motion Extraction

via “act-two performance capture and motion extraction”

AI video generation — Gen-3 Alpha, text/image to video, motion controls, professional filmmaking.

Unique: Act-Two is Runway's proprietary motion capture model, enabling mocap-free motion extraction from video; suggests computer vision approach to skeletal tracking rather than hardware-based capture, but output formats and re-targeting pipeline are undocumented

vs others: Eliminates need for mocap suits or specialized hardware; video-based approach is more accessible than traditional mocap, but accuracy and output quality compared to professional mocap systems unknown

via “single-video cinematic motion extraction”

[ECCV 2024 Oral] MotionDirector: Motion Customization of Text-to-Video Diffusion Models.

Unique: Applies LoRA exclusively to temporal attention layers while freezing spatial layers, forcing the model to learn only motion dynamics without memorizing scene content. Uses auxiliary losses to encourage motion-content disentanglement.

vs others: Extracts pure camera motion without scene-specific artifacts, unlike optical flow-based methods which are sensitive to scene depth and lighting changes.

via “motion reference video analysis and extraction”

magicanimate — AI demo on HuggingFace

Unique: Automatically extracts motion guidance from arbitrary reference videos without requiring manual annotation or pose labeling, using pre-trained vision models to infer motion patterns that generalize across different subjects

vs others: More flexible than keyframe-based animation (no manual specification required) but less precise than explicit motion capture data; faster than manual motion design but slower than pre-computed motion libraries

via “single-character-motion-extraction”

via “multi-person-motion-capture”

via “single-subject-motion-isolation”

via “ai-driven character motion capture and animation”

via “multi-person skeletal tracking and pose detection in single video”

Unique: Automatically detects and separates multiple people in a single video without manual per-person segmentation, enabling efficient capture of group scenes and interactions; outputs distinct FBX files per person, allowing independent character animation and reuse in different contexts

vs others: More efficient than filming each character separately and manually synchronizing animations; more accessible than professional mocap studios which require controlled environments and marker placement on each actor; more flexible than pose libraries which are limited to single-character poses