The Facial Action Coding System (FACS) was
developed by Paul Ekman to categorise human
It works as an automated computer system which
can categorise human emotions according to
changes or movements of the face. Facial
expressions are broken down into the individual
Action Units that make up a specific expression
over time. The output of your collected audience
emotions are aggregated and displayed within the
dashboard as six basic emotion metrics and three
proprietry metrics for measuring the emotional
In addition to our emotion metrics, we also
track a variety of proprietary metrics. These
have been derived through our own research,
and can be used in conjunction with basic
emotions classifiers to gain deeper insights
into the content tested.
We’ve developed our own face detector which is more accurate and reliable than the Viola-Jones detector, generally considered the industry standard.
Once the region of the face has been detected, we need to be able to identify the landmark points of the face: the nose, the eyes, the mouth, the eyebrows.
Using all the frames in respondent’s
video, we create a person-dependent
baseline, or mean face shape. By measur
ing their expressions as they deviate from
their ‘neutral’ face, accounts for people
who naturally look more positive or nega
tive and correcting any bias.
For each frame, the algorithm estimates the position of the face.
49 keypoints are tracked around facial features including the mouth, nose, eyes and eyebrows.
From these keypoints, we extract the face image and feed it to our deep learning based facial interpreter. It will produce a signature encoding the expressions of the face.The model learned by looking at millions of faces.
To encode the dynamics of facial emotions, a temporal module stores those signatures and processes them in a sequence to estimate whether or not the appropriate expression is present.
This frame by frame data is collected to give a trace of when the face is expressing the emotion.
To review the accuracy of our motions, we plot the aggregate output of
our metrics against ground truth data – the majority voting of 5 human
annotators who notate frame by frame the emotion they see.
Bad lighting, heavy shadows, thick facial hair,
glasses, typically makes emotion detection more
challenging. However, we use ‘in the wild’ datasets
that employs machine learning that teach our
algorithms to cope with such obstacles.
Get more information on how we collect
respondent data from our tech white paper.
Face detection Facial landmarks 3D headpose Basic emotions Expressivity Valence Attention
Non-exaggerated natural reactions Complete in-the-wild environment Video watching in mobile and desktop web browsers Precision over speed
Emotional events Emotions duration, intensity, volume Emotions by video segments Emotions by audience segments Interest score Attention volume & quality Memory & emotional hook
Our white paper covers the science behind our facial coding technology and how we’ve taught machines to recognise emotions, just like humans.