Landmark-Guided Motion Transfer for Stable Face Animation
Overview
This repository implements a landmark-guided motion transfer framework for animating a static face image (such as sketches or grayscale faces) using a sequence of driving facial landmarks.
The method estimates dense pixel-wise motion from landmark displacements and refines warped frames using a lightweight U-Net-based generator. Explicit motion warping is followed by frame refinement to reduce artifacts and improve visual stability.
The design emphasizes geometric stability and controlled facial motion, making the framework suitable for sketch-based and low-texture face animation.
Data Preparation
This repository does not include datasets.
For training and evaluation, the pipeline expects:
- A source face image (RGB or grayscale)
- A driving video sequence
- Facial landmarks extracted per frame
- Landmark heatmaps stored as
.npyfiles
All images and frames are resized to a fixed resolution (e.g., 256×256) and normalized before processing.
The method uses a pre-defined driving landmark sequence during inference.
Background
Landmark and keypoint-based motion transfer has been widely explored in face animation research. Early explicit warping methods and later learning-based approaches demonstrated the effectiveness of using sparse motion cues for image animation.
This work follows the general motion transfer paradigm while focusing on semantic landmark guidance and geometric stability.
Key Features
- Landmark-guided dense motion estimation
- Semantic landmark locking for eyes and mouth regions
- Frame-independent synthesis
- Lightweight U-Net refinement
Paper
arXiv link (to be added)
Demo Preview
| Input Image | Generated Animation |
|---|---|
 |
 |
 |
 |
 |
 |