Py-Feat: Python Facial Expression Analysis Toolbox¶

Py-Feat provides a comprehensive set of tools and models to detect, preprocess, analyze, and visualize facial expressions (Action Units, emotions, facial landmarks) from images and videos.

Why you should use Py-Feat¶

Facial expressions convey rich information about how a person is thinking, feeling, and what they are planning to do. Recent innovations in computer vision algorithms and deep learning algorithms have led to a flurry of models that can be used to extract facial landmarks, Action Units, and emotional facial expressions with great speed and accuracy. However, researchers seeking to use these algorithms or tools such as OpenFace, LibreFace, PyAFAR, iMotions-Affectiva, or Noldus FaceReacer may find them difficult to install, use, or too expensive to purchase. It's also difficult to use the latest model or know exactly how good the models are for proprietary tools. We developed Py-Feat to create a free, open-source, and easy to use tool for working with facial expressions data.

Who is it for?¶

Py-Feat was created for two primary audiences in mind: - Human behavior researchers: Extract facial expressions from face images or videos with a simple line of code and analyze your data with Feat. - Computer vision researchers: Develop & share your latest model to a wide audience of users.

Introducing Py-FEAT v2¶

We have released version 2 of Py-Feat that includes a new multi-task face expression model, many bug fixes, performance optimizations, and a standalone app called pyfeat-live. We still support our original version 1 of Py-FEAT that is described in Cheong et al., 2023 and there are advantages to both versions. You can access them as Detectorv1 and Detectorv2. They return the same Fex data structure, so downstream analysis and plotting code is largely shared, but they take different approaches. Both can run on CUDA and MPS gpus and support batching, which greatly accelerates detection.

Two detectors: `Detectorv1` and `Detectorv2`¶

Detectorv1 is a modular pipeline. You choose the model for each stage — face detection, facial landmarks, Action Units, emotions, gaze, and identity — and you can swap any of them or turn one off (pass None). It produces the classic 68-point facial landmarks and runs separate models (e.g. an XGBoost AU classifier, ResMaskNet emotions) one after another. That flexibility comes at a cost: running several models in sequence makes it slower on a single frame.

Detectorv2 runs a single multi-task neural network that predicts Action Units, emotions, valence/arousal, gaze, head pose, a 478-point 3D MediaPipe FaceMesh, and 52 MediaPipe/ARKit blendshapes in one forward pass (plus optional identity embeddings). Because one network replaces the per-task model chain, it is much faster — especially on single frames, has improved performance on action unit and gaze prediction — and adds continuous valence/arousal and gaze that v1 does not produce. The trade-off is that the model set is fixed: you don't pick or disable individual components.

	`Detectorv1` (v1)	`Detectorv2` (v2)
Architecture	modular, one model per task	single multi-task network
Swap / disable models	yes	fixed set
Landmarks	68-point (dlib-style)	478-point 3D MediaPipe FaceMesh
Valence/arousal, gaze	—	built-in
Blendshapes	—	52 MediaPipe/ARKit
Single-frame speed	slower	fast
Best for	specific models, 68-pt conventions, published Cheong et al. benchmarks	speed, video, 3D mesh, valence/arousal + gaze

from feat import Detectorv1, Detectorv2

# v1 — modular: pick or disable models, 68-point landmarks
detector_v1 = Detectorv1(au_model="xgb", emotion_model="resmasknet", identity_model=None)
fex = detector_v1.detect("face.jpg", data_type="image")

# v2 — one fast multi-task model, 478-point 3D mesh + valence/arousal + gaze
detector_v2 = Detectorv2(device="cuda")        # or device="cpu" / "mps"
fex = detector_v2.detect("face.jpg", data_type="image")

Py-feat Live¶

Prefer a graphical app to writing code? Py-feat Live is a native desktop app (macOS, Windows, and Linux) that runs the full Py-Feat pipeline — including a real-time webcam demo — with no code. It wraps the same Detectorv1 and Detectorv2 models, so you get the same Fex outputs you can analyze in Python. Learn more →

License¶

Py-FEAT is provided under the MIT license. You also need to cite and respect the licenses of each model you are using. Note that several models have a non-commercial stipulation. Please see the LICENSE file for links to each model's license information.