Practice Based Research -Lip Sync Animation

• Research on Existing Lip Shape Charts

Many lip-sync mouth shape charts are widely available online, summarizing common mouth positions during speech. Animators often use a set of standard mouth shapes to match various phonemes . These charts work well for English dialogue, but using them directly for Chinese speech can lead to mismatches . Differences in phonetic systems between languages mean some mouth shapes do not correspond exactly. For example, Chinese retroflex sounds (such as “zhi/chi/shi”) have mouth positions that don’t exactly match similar English sounds. Therefore, I first analyzed existing mouth shape charts to understand their strengths, limitations, and applicability in multi-language animation.

• Experiment Using Video Keyframes

Continuous Mandarin speech has considerable slurring, especially in Northern dialects where erhua (retroflex “-r” suffix) and liaison are common. This makes it difficult for a static mouth chart to accurately match certain sounds. To address this, I recorded videos of real speech and extracted keyframes to observe how the mouth shape changes during continuous speech.

The experiment captured detailed mouth shapes during erhua endings and liaised transitions, providing more precise reference data for lip-sync animation.

Overlapping of keyframes to find the position of stressed/weak sounds as well as successive sounds

• Practice-Based Animation Production

Building on the above research, I carried out a practical animation project to apply my optimized lip-sync approach to character dialogue. During production, I implemented several optimizations for mouth matching. For example, I adjusted the timing of mouth shape changes to align precisely with peak pronunciations, and omitted very subtle mouth transitions during rapid speech to maintain smoothness. I also introduced a special mouth shape adjustment for the “-r” erhua endings. Through this hands-on production, I validated my approach and compiled a set of optimized lip-sync techniques, providing guidance for future Chinese lip-sync animation projects.