3DFilmPCA

Abstract

Direct the avatar, in real time

Once a photorealistic avatar exists, an artist needs to direct it. 3DFilmPCA is a rigged 3D Gaussian Splatting system for SMPL-driven, pre-composite action control from monocular video.

Each 3D Gaussian is bound to a triangle on the SMPL mesh and transformed through a local-to-global parameterization, so posing the skeleton moves the splats coherently across frames. A binding-inheritance density strategy adds detail where it's needed, and an area-aware scaling rule keeps splats proportionate as body shape changes. A custom interactive viewer exposes per-joint rotations and PCA shape parameters with immediate visual feedback.

The system supports real-time pose and shape editing, motion transfer onto cinematic sequences, and free-viewpoint rendering for compositing. 3DFilmPCA is Module 2; the companion reconstruction method is GSDHuman.

Editing

Shape & pose, live

Shape editing

PCA-based SMPL shape parameters are exposed as sliders — height, chest, waist, and more. Deformations reflect on the rigged Gaussian avatar in real time, retargeting the body to different builds while preserving appearance.

Pose editing

Joint-level control over the SMPL kinematic tree: select a joint and adjust its X/Y/Z rotation. The change updates the SMPL pose vector and drives the bound Gaussians via linear blend skinning, with immediate feedback.

Interactive viewer

One panel, full control

A custom real-time viewer ties the system together — reporting live metrics and Gaussian count, toggling splat and mesh views, and driving every edit from a single interface.

RenderingScale modifier, vertical field of view, and splat/mesh visibility toggles.
Body posePer-joint Euler rotations across the full SMPL kinematic tree.
Body shapePCA shape parameters — height, chest, waist — applied live.
Time & scaleStep through frames to inspect temporal and spatial coherence.

Compositing

Motion transfer & free-viewpoint

SMPL poses estimated from film clips are transferred onto the trained avatar; foreground segmentation and inpainting recover a clean background for compositing. The rigged Gaussians render from arbitrary angles with consistent appearance.

Contributions

What's new

C1Shape and pose, in real time. The first system to interactively edit both the body shape and the pose of a Gaussian human avatar with live feedback.
C2A custom editing viewer. A responsive interface for inspecting and manipulating avatar attributes — joints, shape, and rendering — as you work.
C3Cinematic robustness. Demonstrated across diverse film sequences and motion styles via SMPL pose transfer and background inpainting.

Citation

BibTeX

@mastersthesis{hsiao2025avatars,
  title   = {Photorealistic Human Avatar Generation and Real-Time
             Pose Editing with Gaussian Splatting},
  author  = {Hsiao, Yi-Cheng},
  school  = {Texas A\&M University},
  year    = {2025},
  month   = aug,
  type    = {Master's thesis},
  url     = {https://nelson-hsiao-0101.github.io/3DFilmPCA/static/YI-CHENG-PRIMARY-2025.pdf}
}

Acknowledgments & funding

Graduate study was supported by a fellowship from Texas A&M University and research funding from the National Science Foundation.

I sincerely thank Xin Li for his guidance and insight throughout this thesis, and my committee members John Keyser and Ann McNamara for their valuable feedback.