Contributing new detectors
Contributing new detectors#
We would love to developers and researchers to contribute new models to Py-Feat so they can gain a wider audience for their work, while allowing end-users to make more informed choices when selecting models for use.
1. Develop a detector#
Currently we have organized our detectors into the following categories:
Action Unit Detectors
Therefore any model you develop should fall into one those categories to be added. If there is a new model category you would like us to implement (e.g. gaze) please let us know on github!
2. Benchmark your detector#
We benchmark each model on a standard datasets used in the field. We employ an honor system that your model should not be using these datasets for training. We can also add new benchmark dataset and results as needed.
Action Unit Detectors: DIFSA Plus
Emotion Detectors: AffNet
Face Detectors: WIDER
Landmark Detectors: 300W
Facepose Detectors: BIWI
3. Add your code to Py-Feat#
Adding a new model to the Py-FEAT toolbox is easy if you are familiar with Python, Github, package development, and follow the steps below.
It can be helpful to install Py-Feat in development mode so that changes to source files are immediately reflected in any scripts or notebooks that import Py-Feat. To do so, after cloning the code base, install Py-Feat using:
pip install -e . For more details see the general contribution guidelines
Pre-trained models in Py-Feat are organized into sub-folders in the source code based on the detector type:
feat/ au_detectors/ emo_detectors/ face_detectors/ facepose_detectors/ landmark_detectors/
Create a folder for the model you are adding in the appropriate model sub-directory
Add your model code which can be a single
.pyfile ending in
feat/landmark_detectors/mobilefacenet_test.py) or a separate sub-directory containing at least 3 files one of which ends in
_modeland the other that ends in
feat/au_detectors/JAANETfor an example):
__init__.py(this can be empty)
mynewmodel_model.py(this should end in
mynewmodel_test.py(this should end in
Your model should be a class that has the appropriate method that a
Detectorcan call. For example, Emotion detectors should have the method
mynewmodel.detect_emotions()that can be called:
class myNewModel(): ## code to init and load model detect_emotions(self, imgs, *args **kwargs): ## code to detect emotions return [array with probabilities for 7 emotions]
feat/pretrained.py, add your model to the
PRETRAINED_MODELSdictionary. If you’re adding a new AU detector and it only supports specific AUs make sure to also add it to the
AU_LANDMARK_MAP. You may also want to train a new
PLSvisualization model and save it as
.h5file. See this tutorial for details.
Upload your trained model weights to an accessible locations (e.g. Google Drive) and add it to
Follow the general contribution guidelines to add tests and format your code
When your tests pass create a pull-request against the
mainbranch on github!
If you enjoy developing/testing in jupyter notebooks, it can be helpful to add the following lines of code into a cell at the top of yoru notebook so that source code changes don’t require you to restart the kernel: