3d Facial Animation: FACS, Phoneme, Visime
3d Facial Animation Expressions
The human face is perhaps the most advanced communication tool evolved by nature. Consisting of a multitude of highly dynamic muscle connections, this communication device is used to convey emotion and offer feedback to the emotions and actions of others.
Capturing the subtle nuances of the human face is not an easy task, yet throughout the years, psychologists and social science researchers have managed to collect and categorize a comprehensive taxonomy and etymology of muscle expressions into systems.
We'll take a closer look at some of these systems to determine the best option for your next 3d facial animation project.
...
3d Facial Animation Systems
Facial animation systems vary with the complexity of their parts, or abstract units, which in turn reflects the degree of control one might have over each system.
The main three systems within the computer animation domain are phoneme, visime, and the facial action unit system (FACS), with phoneme being the most basic and FACS as the most advanced.
Each 3d facial animation system has its own strengths, weaknesses, learning curve and requires a different set-up time, yet all share a single shape control mechanism.
...
Shape Control Mechanisms
The shape control mechanisms that are most prevalent in 3D animation are blend shapes and morph targets. Morphing is a technique that combines multiple shapes into a single expression or what is called an absolute target, blending between them or morphing one into the other. There is nothing special about these mechanisms, as they are essentially an identical way of mixing Vertex Coordinates between shapes of similar geometric structure.
In my view, these mechanisms are erroneously named differently within various 3D application packages (Autodesk Maya, 3D Studio Max, etc.), which sometimes makes it hard for animators using different 3D applications to communicate with one another. I hope that at some point this naming difference will be reconciled because it creates confusion among people entering the field of 3D Animation.
The other Shape Control Mechanism is Bones and Joints, but since this subject is too broad we won't discuss it just yet. Now on to actual systems.
...
FACS
The use of FACS in 3d facial animation is relatively recent, yet it has gathered a strong momentum and is being used everywhere from feature films to fully animated CG movies. This is purely due to the flexibility of the system and how suitable it is for large projects, where improvements and changes are prevalent.
As FACS is the most advanced facial animation system, the complexity of the inner system may grow exponentially, yet the complexity of controls for the animator will be limited to only what is required. The base facial expression can be overlaid with emotion or a set of mixed emotions, all acting on top of one another as modifiers and correctors to compose the final expression and deliver the most impact.
Using FACS is the way to go if you're looking for a future-proof way to animate a face.
...
Visimes
A visime system, on the other hand, is a system of shapes that produces seemingly similar sounds with the added benefit of emotion control.
However, emotions that overlay sound shapes are still absolute in the way shapes are made.
Read more about this in our section on Visimes System.
...
Phonemes
A phoneme system is the most basic of all, yet it is very powerful on a tight budget. The majority of the time, this is what your project will require, as no actor can isolate expressions from other facial movements.
With phonemes we would scan individual face poses and output them as ready to use products.
In reality, FACS is an approach. You cannot 3D-scan facial action units; you can only extract these units mathematically or statistically if you were to do it the right way.
...
Corrective Morphs
Another use for 3D Scanning is to correct faulty deformation weights with the shape data from a 3D scan, which saves time if the project does not warrant elaborate muscle simulation rigs.
Using 3D scans as corrective blends, in this case, is highly desirable as the muscle volume gets preserved.
The face is the most important part of an actor's body and dedicating a substantial amount time to 3d facial animation will certainly make your work stand out from the rest!
...
FACS and Photogrammetry Combined
Combining the power of FACS metadata and photogrammetry gives VFX producers, supervisors, and CG sups a qualitative edge over their competitors.
The precision of 3D scanned data makes it easy for fans to recognize their idols, which ultimately reinforces the brand and strengthens the story narrative.
With the development of computer processing power, algorithms, and vision sensors, it became possible to 3D scan, mathematically extract, and categorize facial microexpressions and shapes from a collection of photos through photogrammetry.
...
If you have any wishes, suggestions or feel this article could be improved, please, do not hesitate to leave us your comments in the section below.
Best Regards,
Ruslan Vasylev
Founder
Professional 3D Modeler with 17 years of combined VFX and Games experience, managing Vancouver-based 3D Scanning Studio,
Servicing Visual Effects for Film, Television, Games and Virtual Reality production studios.