Py-Feat: Python Facial Expression Analysis Toolbox

Py-Feat: Python Facial Expression Analysis Toolbox#

arXiv-badge Package versioning Tests Coverage Status Python Versions GitHub forks GitHub stars DOI

Py-Feat provides a comprehensive set of tools and models to easily detect facial expressions (Action Units, emotions, facial landmarks) from images and videos, preprocess & analyze facial expression data, and visualize facial expression data.

Why you should use Py-Feat#

Facial expressions convey rich information about how a person is thinking, feeling, and what they are planning to do. Recent innovations in computer vision algorithms and deep learning algorithms have led to a flurry of models that can be used to extract facial landmarks, Action Units, and emotional facial expressions with great speed and accuracy. However, researchers seeking to use these algorithms or tools such as OpenFace, iMotions-Affectiva, or Noldus FaceReacer may find them difficult to install, use, or too expensive to purchase. It’s also difficult to use the latest model or know exactly how good the models are for proprietary tools. We developed Py-Feat to create a free, open-source, and easy to use tool for working with facial expressions data.

Who is it for?#

Py-Feat was created for two primary audiences in mind:

  • Human behavior researchers: Extract facial expressions from face images or videos with a simple line of code and analyze your data with Feat.

  • Computer vision researchers: Develop & share your latest model to a wide audience of users.

and anyone else interested in analyzing facial expressions!

Check out a recent presentation by one of the project leads Eshin Jolly, PhD for a broad-overview and introduction:

Installation#

You can easily install the latest stable version from PyPi:

pip install py-feat

For other installation methods (e.g. Google Collab, development) see the how to install page

Check out our FAQS for common issues and solutions.

Two detectors: Detector and Detectorv2#

Py-Feat ships two detectors. They return the same kind of Fex data structure, so downstream analysis and plotting code is largely shared, but they take different approaches.

Detector (v1) is a modular pipeline. You choose the model for each stage — face detection, facial landmarks, Action Units, emotions, and identity — and you can swap any of them or turn one off (pass None). It produces the classic 68-point facial landmarks and runs separate models (e.g. an XGBoost AU classifier, ResMaskNet emotions) one after another. That flexibility comes at a cost: running several models in sequence makes it slower on a single frame.

Detectorv2 (v2) runs a single multi-task neural network that predicts Action Units, emotions, valence/arousal, gaze, head pose, and a 478-point 3D MediaPipe FaceMesh in one forward pass (plus face detection and optional identity embeddings). Because one network replaces the per-task model chain, it is much faster — especially on single frames — and adds continuous valence/arousal and gaze that v1 does not produce. The trade-off is that the model set is fixed: you don’t pick or disable individual components.

Detector (v1)

Detectorv2 (v2)

Architecture

modular, one model per task

single multi-task network

Swap / disable models

✅ yes

❌ fixed set

Landmarks

68-point (dlib-style)

478-point 3D MediaPipe FaceMesh

Valence/arousal, gaze

—

✅ built-in

Single-frame speed

slower

fast

Best for

specific models, 68-pt conventions, published Cheong et al. benchmarks

speed, video, 3D mesh, valence/arousal + gaze

 
from feat import Detector, Detectorv2

# v1 — modular: pick or disable models, 68-point landmarks
detector = Detector(au_model="xgb", emotion_model="resmasknet", identity_model=None)
fex = detector.detect("face.jpg", data_type="image")

# v2 — one fast multi-task model, 478-point 3D mesh + valence/arousal + gaze
detector_v2 = Detectorv2(device="cuda")        # or device="cpu" / "mps"
fex = detector_v2.detect("face.jpg", data_type="image")

Available models#

Py-feat includes several pre-trained models for Action Unit detection, Emotion detection, Face detection, Facial Landmark detection, and Face/Head post estimation.

You can check out the full list on the pre-trained models page.

Contributions#

We are excited for people to add new models and features to Py-Feat. Please see the contribution guides.

License#

Py-FEAT is provided under the MIT license. You also need to cite and respect the licenses of each model you are using. Please see the LICENSE file for links to each model’s license information.

Contents