Initialize a dense skeleton
Voxelize the mesh, extract its medial structure, and build a tree of candidate joints. RBF skinning connects the mesh to this initial skeleton.
Under review / 2026
Rig-from-Motion
RfM: Zero-shot rigging
from video
diffusion models
Jointly optimizing skeletal structure,
skinning, and animation from video priors.
1 Georgia Institute of Technology · 2 Ant Group
Animation and learned skeletons
Static mesh → Motion prior → Articulated rig
Static mesh + text prompts → skeleton, skinning, and animationOverview
01 — OverviewRigging from motion priors
Zero-shot rigging
from video
motion priors
RfM uses motion priors from a pretrained video diffusion model to optimize an articulated 3D skeleton.
Given a static mesh and text motion prompts, Rig-from-Motion jointly optimizes skeletal structure, skinning, and animation using a frozen video diffusion model. The method requires neither a manually rigged training set nor a fixed skeleton template.
A stochastic skeleton assigns an existence probability to each candidate joint. Motion-driven optimization jointly refines the structure and kinematics to produce a sparse articulated rig.
01
Zero-shot rigging
Uses pretrained motion priors without training on manually rigged examples.
02
Structure from motion
Learns which joints to keep alongside the motion they need to express.
03
Diverse mesh categories
Evaluated on humanoids, quadrupeds, aquatic creatures, and unusual forms.
02 — ResultsSupplementary animations
Animation and
rigging results
Generated animations and learned skeletons. Select an example to view its result and text prompt.
Example 01 / 11Animation
RfM outputRendered animation · 256 × 256
Humanoid
The astronaut
“The astronaut is walking.”
Rendered results from the supplementary material. Skeleton overlays show the learned articulation.
03 — MethodJoint structure and motion optimization
Method
overview
RfM optimizes skeletal topology, skinning, and kinematics through video diffusion supervision.
Distill a motion prior
Render motion from sampled viewpoints. Text-conditioned video score distillation guides the animation, while physical regularization improves its plausibility.
Optimize a sparse rig
Hard Concrete gates learn each joint’s activation. Sparsity and symmetry regularization refine the structure; pruning produces the final control skeleton.
Read the abstract
We present Rig-from-Motion (RfM), a zero-shot framework for automatic rigging and physically plausible free-view animation rendering. Unlike methods constrained by large, manually rigged datasets, RfM uses motion priors from a pretrained video diffusion model to animate a static mesh from text prompts while generating a fine-grained skeleton. Optimization is driven by a Score Distillation Sampling loss. A stochastic skeleton representation associates each potential joint with an existence probability, allowing the Hard Concrete distribution to optimize skeletal topology and kinematics jointly. Sparsity, self-intersection, and gravity regularization help discover a sparse, plausible skeletal structure. Qualitative and quantitative results support RfM as an alternative to conventional rigging techniques that does not require rigged training data.
Condensed from the paper. See the PDF for the complete abstract and formulation.
04 — Physical plausibilityRegularization ablations
Regularization
ablations
Gravity and self-intersection regularization improve ground contact and reduce intersections between articulated parts.
Without gravity regularization, the wolf’s hind legs float. The full model maintains more plausible contact with the ground.
05 — EvaluationSkeleton and animation quality
Quantitative
evaluation
Evaluated on 44 meshes: 24 sampled from RigXL and 20 out-of-distribution assets.
4.15/5
Anatomical correctness
OOD user study · 18 raters0.84
Physical commonsense
VideoPhy · higher is better1 hour
Optimization per mesh
Single NVIDIA A100 · 40 GBSkeleton quality on unseen shapes
20 OOD meshes · 18 raters · Scores from 1 to 5 · Higher is better
| Method | Completeness ↑ | Conciseness ↑ | Correctness ↑ |
|---|---|---|---|
| RigNet | 2.69 | 2.53 | 2.50 |
| UniRig | 2.73 | 3.23 | 2.89 |
| MagicArticulate | 3.45 | 3.27 | 3.24 |
| Puppeteer | 3.32 | 3.58 | 3.48 |
| AnyMate | 3.54 | 1.84 | 1.90 |
| RfM Ours | 3.97 | 4.13 | 4.15 |
Animation quality and runtime
VideoPhy semantic adherence (SA) and physical commonsense (PC)
| Method | SA ↑ | PC ↑ | Time per sample ↓ |
|---|---|---|---|
| Puppeteer | 0.21 | 0.55 | 1 h |
| AKD | 0.34 | 0.79 | 25 h |
| RfM Ours | 0.37 | 0.84 | 1 h |
Values reported in the paper. All runtimes measured on one A100 40 GB. RfM optimizes each mesh and is slower than feed-forward rigging methods. See the paper for RigXL, skinning, and additional ablation results.
Limitations
RfM inherits biases from its video prior and can miss articulation when motion prompts fail to probe a functional part. Stronger motion priors can help, at a higher computational cost. Its per-mesh optimization also limits throughput.
Citation
@misc{tao2026rfm,
title = {RfM: Zero-Shot Rigging from
Video Diffusion Models},
author = {Tao, Shuohan and Zheng, Shunyuan
and Zhou, Boyao},
year = {2026},
note = {Under review},
url = {https://shuohantao.github.io/rfm-project/}
}