How do you use animation graph and animation blueprint to control facial expressions and lip sync?

1 Animation Graph

Animation graph is a visual scripting system that lets you define the flow of animations for your character. You can use nodes to connect different animation assets, such as poses, blendspaces, montages, and curves. You can also use nodes to control the transitions, conditions, and parameters of your animations. For example, you can use a state machine node to switch between different animation states based on your character's speed, direction, or input. Animation graph is part of the animation blueprint, which is a blueprint class that inherits from the anim instance class.

2 Animation Blueprint

Animation blueprint is a blueprint class that controls the animation logic and data for your character. It contains the animation graph, as well as variables, functions, and events that you can use to communicate with other blueprints and systems. For example, you can use animation blueprint to get the character's movement, health, or inventory data, and use them to modify the animation graph. You can also use animation blueprint to trigger events, such as sound effects, particles, or logic, based on the animation state or timing. Animation blueprint is assigned to the skeletal mesh component of your character.

3 Facial Expressions

Facial expressions are essential for conveying your character's emotions, personality, and reactions. To create facial expressions, you need to have a skeletal mesh with facial bones or blend shapes that can deform the face. You also need to have animation assets that can drive the facial bones or blend shapes, such as poses, curves, or morph targets. You can use animation graph to blend different facial expressions based on your character's mood, situation, or dialogue. For example, you can use a blend by int node to switch between different facial poses based on an integer variable that represents the emotion. You can also use a blend by curve node to blend between different facial curves based on a float variable that represents the intensity.

4 Lip Sync

Lip sync is the process of synchronizing your character's mouth movements with the spoken dialogue. To create lip sync, you need to have a skeletal mesh with mouth bones or blend shapes that can open and close the mouth. You also need to have audio assets that contain the dialogue and the phonemes, which are the basic units of speech sounds. You can use animation graph to blend different mouth shapes based on the phonemes of the audio. For example, you can use a pose by name node to play a mouth pose based on the name of the phoneme. You can also use a pose driver node to drive a mouth blend shape based on the value of the phoneme.

5 Animation Graph vs Animation Blueprint

Animation graph and animation blueprint are both important for creating facial animations and lip sync, but they have different roles and functions. Animation graph is the visual scripting system that defines the flow and logic of animations, while animation blueprint is the blueprint class that contains the animation graph and other data and functions. Animation graph is where you blend and control the animation assets, such as facial poses, curves, or morph targets, while animation blueprint is where you get and set the variables, functions, and events that affect the animation graph, such as emotion, intensity, or phoneme. Animation graph and animation blueprint work together to create dynamic and expressive facial animations and lip sync.

6 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?