Caroline Pacheco do E.Silva (Ph.D.)'s profile

Facial Expression Public Databases

Last update: 09/04/2018


Here are the main databases to evaluate the facial expression recognition algorithms. We shown a brief description and links to download each database. The databases are organized by alphabetically order.

1. 3D Twins Expression Challenge (3D-TEC) 

This database contains 3D face scans for 107 pairs of twins. There are 107 x 2 = 214 individuals, each with a 3D face scan with a smiling expression and a scan with a neutral expression, and so 214 x 2 = 428 scans. The scans were acquired with a Minolta Vivid 910. 

More information about the database can be found here.

Link: https://sites.google.com/a/nd.edu/public-cvrl/data-sets
2. AffectNet (2017)                                                   

AffectNet contains more than 1M facial images collected from the Internet by querying three major search engines using 1250 emotion related keywords in six different languages. About half of the retrieved images (~440K) were manually annotated for the presence of seven discrete facial expressions (categorial model) and the intensity of valence and arousal (dimensional model). AffectNet is by far the largest database of facial expressions, valence, and arousal in the wild enabling research in automated facial expression recognition in two different emotion models. Two baseline deep neural networks are used to classify images in the categorical model and predict the intensity of valence and arousal. Various evaluation metrics show that our deep neural network baselines can perform better than conventional machine learning methods and off-the-shelf facial expression recognition systems.

More information about the database can be found here.

3. Affectiva-MIT Facial Expression Dataset (AM-FED) (2013)                                                   

This dataset consists of 242 facial videos (168,359 frames) recorded in real world conditions. The database is descripted as follows:
1) Frame- by-frame labels for:
    a) 10 symmetrical FACS action units;
    b) 4 asymmetric (unilateral) FACS action units;
    c) 2 head movements, smile, general expressiveness, feature tracker fails;
    d) Gender.
2) The location of 22 automatically detected landmark points.
3) Self-report responses of familiarity with, liking of, and desire to watch again for the stimuli videos.
4) Baseline performance of detection algorithms on this dataset. We provide baseline results for smile and AU2 (outer eyebrow raise) on this dataset using custom AU detection algorithms.

More information about the database can be found here.

4. A Natural Visible and Infrared facial Expression Database (2010)

The Key Laboratory of Computing and Communication Software of Anhui Province(CCSL) has constructed the USTC-NVIE (Natural Visible and Infrared facial Expression) database under the sponsor of the 863 project. To date, most facial expression analysis has been based on visible and posed expression databases. Visible images, however, are easily affected by illumination variations, while posed expressions differ in appearance and timing from natural ones. We propose and establish a natural visible and infrared facial expression database, which contains both spontaneous and posed expressions of more than 100 subjects, recorded simultaneously by a visible and an infrared thermal camera, with illumination provided from three different directions. The posed database also includes expression image sequences with and without glasses.

More information about the database can be found here.
5. AR Face (1998)

This face database was created by Aleix Martinez and Robert Benavente in the Computer Vision Center (CVC) at the U.A.B. It contains over 4,000 color images corresponding to 126 people's faces (70 men and 56 women). It contains frontal view faces with different facial expressions, illumination conditions, and occlusions (sun glasses and scarf). The pictures were taken at the CVC under strictly controlled conditions. No restrictions on wear (clothes, glasses, etc.), make-up, hair style, etc. were imposed to participants. Each person participated in two sessions, separated by two weeks (14 days) time. The same pictures were taken in both sessions.

More information about the database can be found here.

6. Belfast Naturalistic (2003)

The Belfast Naturalistic Emotional Database consists of  298 audiovisual clips from 125 speakers, 31 male, 94 female. Emotional clips are episodes which appear to provide within themselves at least most of the context necessary to understand a local peak in the display of  emotion and to show how it develops over time. For each speaker there is at least one clip showing him or her in a state judged relatively emotional, and also one clip in a state that the selector judged relatively neutral. Clips range  from 10-60 secs in length. The clips are stored as MPEG files, with audio data extracted into .wav files 

More information about the database can be found here.

Link: http://belfast-naturalistic-db.sspnet.eu/
7. Binghamton University Facial Expression Databases: BP4D-Spontanous Database, BU-3DFE Database (Static Data), BU-4DFE Database (Dynamic Data)

The Binghamton University was materialistic in creating 3D facial expression databases for the purpose of  evaluation of algorithms. The databases come in two versions, one with the static data and the other with dynamic data. The static database includes still color images, while the dynamic database contains video sequences of subjects with expressions. The databases also include the neutral expression with the six prototypic expressions.

I. BU-3DFE (Binghamton University 3D Facial Expression) Database (Static Data) (2006)
3D facial models have been extensively used for 3D face recognition and 3D face animation, the usefulness of such data for 3D facial expression recognition is unknown [Yin et al. (2006)]. This 3D facial expression database (called 
BU-3DFE database) includes 100 subjects with 2500 facial expression models. The BU-3DFE database is available to the research community (e.g., areas of interest come from as diverse as affective computing, computer vision, human computer interaction, security, biomedicine, law-enforcement, and psychology). The database presently contains 100 subjects (56% female, 44% male), ranging from 18 to 70 years of age, with a 
variety of ethnic/racial ancestries, including White, Black, East-Asian, Middle-east Asian, Indian, and Hispanic  Latino. Participants in face scans include undergraduates, graduates and faculty from our institute’s departments of  Psychology, Arts, and Engineering (Computer Science, Electrical Engineering, System Science, and Mechanical 
Engineering). The majority of participants were undergraduates from the Psychology Department (collaborator: Dr. 
Peter Gerhardstein). Each subject performed seven expressions in front of the 3D face scanner (see right of figure 14). With the exception of the neutral expression, each of the six prototypic expressions (happiness, disgust, fear, angry, surprise and sadness) includes four levels of intensity (see left of figure 14). Therefore, there are 25 instant 3D expression models for each subject, resulting in a total of 2,500 3D facial expression models in the database. Associated with each expression shape model, is a corresponding facial texture image captured at two views (about +45° and -45°). As a result, the database consists of 2,500 two-view’s texture images and 2,500 geometric shape models

More information about the database can be found here.

II. BU-4DFE (3D + time): A 3D Dynamic Facial Expression Database (Dynamic Data) (2008)
To analyze the facial behaviour from a static 3D space to a dynamic 3D space, the BU-3DFE was extended to the  BU-4DFE. A newly created high-resolution 3D dynamic facial expression database is available to the scientific  research community [Yin et al. (2008)]. The 3D facial expressions are captured at a video rate (25 frames per  second). For each subject, there are six model sequences showing six prototypic facial expressions (anger, disgust, happiness, fear, sadness, and surprise), respectively. Each expression sequence contains about 100 frames (a sample seen in figure 16). The database contains 606 3D facial expression sequences captured from 101 subjects, with a total of approximately 60,600 frame models. Each 3D model of a 3D video sequence has the resolution of approximately 35,000 vertices (see figure 15). The texture video has a resolution of about  1040×1329 pixels per frame. The resulting database consists of 58 female and 43 male subjects, with a variety of ethnic/racial ancestries, including Asian, Black, Hispanic/Latino, and White. This database includes the salient features of the 3D database in the previous sub-section along with the dynamic characteristics


More information about the database can be found here.

8. Biwi 3D Audiovisual Corpus of Affective Communication (2010)
 
The corpus contains high quality dynamic (25 fps) 3D scans of faces recorded while pronouncing a set of English sentences. Affective states were induced by showing emotional video clips to the speakers. The data has been annotated by tracking all frames using a generic face template, segmenting the speech signal into single phonemes, and evaluating the emotions conveyed by the recorded sequences by means of an online survey.

More information about the database can be found here.

Link: http://www.vision.ee.ethz.ch/datasets/b3dac2.en.html
9. Cohn-Kanade AU-Coded Facial Expression (2000)

Version 1, the initial release, includes 486 sequences from 97 posers. Each sequence begins with a neutral expression and proceeds to a peak expression. The peak expression for each sequence in fully FACS coded and given an emotion label. The emotion label refers to what expression was requested rather than what may actually have been performed. For a full description of CK, see (Kanade, Cohn, & Tian, 2000).For validated emotion labels, please use version 2, CK+, as described below.

Version 2, referred to as CK+, includes both posed and non-posed (spontaneous) expressions and additional types of metadata. For posed expressions, the number of sequences is increased from the initial release by 22% and the number of subjects by 27%. As with the initial release, the target expression for each sequence is fully FACS coded. In addition validated emotion labels have been added to the metadata. Thus, sequences may be analyzed for both action units and prototypic emotions. The non-posed expressions are from Ambadar, Cohn, & Reed (2009). Additionally, CK+ provides protocols and baseline results for facial feature tracking and action unit and emotion recognition. 

More information about the database can be found here.

10. Dynamic and spontaneous emotional facial expression database (DynEmo) (2013)        

The DynEMo is a database available to the scientific community. It contains dynamic and natural emotional facial expressions (EFEs) displaying subjective affective states rated by both the expresser and observers. Methodological and contextual information is provided for each expression. This multimodal corpus meets psychological, ethical, and technical criteria. It is quite large, containing two sets of 233 and 125 recordings of EFE of ordinary Caucasian people (ages 25 to 65, 182 females and 176 males) filmed in natural but standardized conditions.  filmed in natural but standardized conditions. In the Set 1, EFE recordings are associated with the affective state of the expresser (self-reported after the emotion inducing task, using dimensional, action readiness, and emotional labels items). In the Set 2, EFE recordings are both associated with the affective state of the expresser and with the time line (continuous annotations) of observers’ ratings of the emotions displayed throughout the recording. The time line allows any researcher interested in analysing non-verbal human behavior to segment the expressions into small emotion excerpts. 

More information about the database can be found here.

11. Denver Intensity of Spontaneous Facial Actions (DISFA) (2013)                                       
 
This database contains stereo videos of 27 adult subjects (12 females and 15 males) with different ethnicities. The video of each subject was acquired using PtGrey stereo imaging system (1024x768 resolution @ 20 fps) while each subject watching a 4-minute emotive video stimulus. The intensities of 12 FACS Action Units defined in 0-5 scale were manually coded frame by frame by a human FACS expert. The reliability of manual coding was evaluated by a second FACS coder. The database also includes 66 facial landmark points of each image in the database. The database is available for distribution for research purposes.
 
More information about the database can be found here.
 
Link: http://www.engr.du.edu/mmahoor/DISFA.htm
12. EURECOM Kinect Face (2014)                                                                                             

This databas econsists of the multimodal facial images of 52 people (14 females, 38 males) obtained by Kinect. The data is captured in two sessions happened at different time period (about half month). In each session, the dataset provides the facial images of each person in 9 states of different facial expressions, different lighting and occlusion conditions: neutral, smile, open mouth, left profile, right profile, occlusion eyes, occlusion mouth, occlusion paper and light. 

More information about the database can be found here.

Link: http://rgb-d.eurecom.fr/
13. Extended Yale Face Database B (B+) (2001)

The extended Yale Face Database B contains 16128 images of 28 human subjects under 9 poses and 64 illumination conditions. The data format of this database is the same as the Yale Face Database B.

More information about the database can be found here.
14. Facial Expression In Wild (2011 and 2012)

I. Acted Facial Expressions in the Wild (AFEW) (2012) is a dynamic temporal facial expressions data corpus consisting of close to real world environment extracted from movies. More information about the AFEW can be found here.

II. Static Facial Expressions in the Wild (SFEW) (2011) has been developed by selecting frames from AFEW.  More information about the SFEW can be found here.

Link: https://cs.anu.edu.au/few/AFEW.html
15. GFT Facial Expression Database (2017)

The Sayette Group Formation Task (GFT) Spontaneous Facial Expression Database has two components: meta-data and video data. Because video, by its very nature, reveals person identity, a separate procedure is needed for its distribution. The video data is available from the University of Pittsburgh. The meta-data includes baseline results, MATLAB functions, frame-level annotations, and a wiki that describes the formatting and recommended analysis strategies for the data. The GFT database addresses the need for well-annotated video of multiple participants during unscripted interactions. The database includes 172,800 video frames from 96 participants in 32 three-person groups. To aid in the development of automated facial expression analysis systems, GFT includes expert annotations of FACS occurrence and intensity, facial landmark tracking, and baseline results for linear SVM, deep learning, active patch learning, and personalized classification. Baseline performance is quantified and compared using identical partitioning and a variety of metrics (including means and confidence intervals).

More information about the database can be found here.

Link: https://osf.io/7wcyz/wiki/home/
16. Indian Movie Face database (IMFDB) (2013)                                                                        

The IMFDB is a large unconstrained face database consisting of 34512 images of 100 Indian actors collected from more than 100 videos. All the images are manually selected and cropped from the video frames resulting in a high degree of variability interms of scale, pose, expression, illumination, age, resolution, occlusion, and makeup. IMFDB is the first face database that provides a detailed annotation of every image in terms of age, pose, gender, expression and type of occlusion that may help other face related applications.

More information about the database can be found here.

Link: http://cvit.iiit.ac.in/projects/IMFDB/
17. Indian Spontaneous Expression Database (ISED) (2016)

The  ISED  contains  near frontal  face  recordings  of  spontaneous  emotions  at  high resolution and frame rates along with information regarding  gender  of  the  participants,  the  ground-truth  of  emotional clips  and  its  intensity,  and  the peak  emotion intensity frame  in the video clips.
Head movement in all directions werere also allowed. The covered emotions include Happiness, Disgust, Sadness, and Surprise. 

More information about the database can be found here.

18. Japanese Female Facial Expression (JAFFE) (1998)

This database contains 213 images of 7 facial expressions (6 basic facial expressions + 1 neutral) posed by 10 Japanese female models. Each image has been rated on 6 emotion adjectives by 60 Japanese subjects. The database was planned and assembled by Michael Lyons, Miyuki Kamachi, and Jiro Gyoba. We thank Reiko Kubota for her help as a research assistant. The photos were taken at the Psychology Department in Kyushu University.

More information about the database can be found here.

19. Karolinska Directed Emotional Faces (KDEF) (1998)
The Karolinska Directed Emotional Faces (KDEF) is a set of totally 4900 pictures of human facial expressions of emotion. The material was developed in 1998 by Daniel Lundqvist, Anders Flykt and Professor Arne Öhman at Karolinska Institutet, Department of Clinical Neuroscience, Section of Psychology, Stockholm, Sweden.
The material was originally developed to be used for psychological and medical research purposes. More specifically material was made to be particularly suitable for perception, attention, emotion, memory and backward masking experiments. Hence, particular attention was for instance paid to create a soft, even light, shooting expressions in multiple angles, use of uniform T-shirt colors, and use of a grid to center participants face during shooting, and positioning of eyes and mouths in fixed image coordinates during scanning.
The set contains 70 individuals, each displaying 7 different emotional expressions, each expression being photographed (twice) from 5 different angles. 

More information about the database can be found here.
20. MMI Facial Expression (2005)

The database consists of over 2900 videos and high-resolution still images of 75 subjects. It is fully annotated for the presence of AUs in videos (event coding), and partially coded on frame-level, indicating for each frame whether an AU is in either the neutral, onset, apex or offset phase. A small part was annotated for audio-visual laughters. The database is freely available to the scientific community.

More information about the database can be found here.

21. Natural Visible and Infrared facial Expression (NVIE) (2010)

The database contains both spontaneous and posed expressions of more than 100 subjects, recorded simultaneously by a visible and an infrared thermal camera, with illumination provided from three different directions. The posed database also includes expression image sequences with and without glasses.

More information about the database can be found here.
22. ND-2006 Data Set (2005)

The database contains a total of 13,450 images showing six different types of expressions (Neutral, Happiness, Sadness, Surprise, Disgust, and Other). There are images of 888 distinct persons, with as many as 63 images per subject, in this database.

More information about the database can be found here.
23. Radboud Faces (2010)

The Radboud Faces Database (RaFD) is a set of pictures of 67 models (including Caucasian males and females, Caucasian children, both boys and girls, and Moroccan Dutch males) displaying 8 emotional expressions. The RaFD in an initiative of the Behavioural Science Institute of the Radboud University Nijmegen, which is located in Nijmegen (the Netherlands), and can be used freely for non-commercial scientific research by researchers who work for an officially accredited university.
The RaFD is a high quality faces database, which contain pictures of eight emotional expressions. Accordingly to the Facial Action Coding System, each model was trained to show the following expressions: Anger, disgust, fear, happiness, sadness, surprise, contempt, and neutral. Each emotion was shown with three different gaze directions and all pictures were taken from five camera angles simultaneously.

More information about the database can be found here.

Link: http://www.rafd.nl
24. Senthilkumar Face Database Version 1 

The Senthil face database contains 80 face images (the website contradicts itself as to whether they are color or black and white) of 5 men. There are frontal views of the faces with different facial expressions, occlusions and brightness conditions. There 16 images of each person.
25. SN-Flip Crowd Video Data

Comprising 190 subjects recorded in 28 crowd videos over a two year period, SN-Flip captures variations in illumination, facial expression, scale, focus, and pose.  The videos were recorded with point-and-shoot camcorders from the Cisco Flip family of products, so the image quality is representative of typical web videos.  Ground truth information for subject identities and social groups is included to facilitate future research in vision-driven social network analysis.  To obtain this data set, retrieve the license agreement.  

The more details can be found here
26. The Bimodal Face and Body Gesture (FABO) (2006)

The Bimodal Face and Body Gesture Database (FABO) for Automatic Analysis of Human Nonverbal Affective Behavior was created at the Faculty of Information Technology at the University of Technology, Sydney (UTS) by Hatice Gunes and Massimo Piccardi in 2005. Posed visual data was collected from volunteers in a laboratory setting by asking and directing the participants on the required actions and movements. The FABO database contains videos of face and body expressions recorded by the face and body cameras, simultaneously, as shown in the figures below. This database is the first to date to combine facial and body displays in a truly bimodal manner, hence enabling significant future progresses in affective computing research.

The details of FABO can be found here.

27. The Bosphorus Database (2008)

The Bosphorus Database is intended for research on 3D and 2D human face processing tasks including expression recognition, facial action unit detection, facial action unit intensity estimation, face recognition under adverse conditions, deformable face modeling, and 3D face reconstruction. There are 105 subjects and 4666 faces in the database. This database is unique in three aspects:

1) Rich repertoire of expressions:
                       Up to 35 expressions per subject
                       FACS scoring (includes intensity and asymmetry codes for each AU)
                       One third of the subjects are professional actors/actresses
2) Systematic head poses (13 yaw and pitch rotations)

3) Varieties of face occlusions (beard & moustache, hair, hand, eyeglasses)

More information about the database can be found here.

28. The Child Affective Face Set (CAFE) (2015)                                                                 

The Child Affective Facial Expressions Set (CAFE) is the first attempt to create a large and representative set of children making a variety of affective facial expressions that can be used for scientific research in this area. The set is made up of 1200 photographs of over 100 child models (ages 2-8) making 7 different facial expressions - happy, angry, sad, fearful, surprise, neutral, and disgust. 

More information about the database can be found here.

Link: http://childstudycenter.rutgers.edu/Child_Affective_Facial_Expression_Set.html
29. The CMU Multi-PIE Face (2009) 

The CMU Multi-PIE face database contains more than 750,000 images of 337 people recorded in up to four sessions over the span of five months. Subjects were imaged under 15 view points and 19 illumination conditions while displaying a range of facial expressions. In addition, high resolution frontal images were acquired as well. In total, the database contains more than 305 GB of face data.

More information about the database can be found here.

30. The Color FERET Database, USA

The FERET database was collected in 15 sessions between August 1993 and July 1996. The database contains 1564 sets of images for a total of 14,126 images that includes 1199 individuals and 365 duplicate sets of images. A duplicate set is a second set of images of a person already in the database and was usually taken on a different day. For some individuals, over two years had elapsed between their first and last sittings, with some subjects being photographed multiple times. This time lapse was important because it enabled researchers to study, for the first time, changes in a subject's appearance that occur over a year.

More information about the database can be found here.

31. The EURECOM Kinect Face Dataset (EURECOM KFD) (2014)                                    

The database consists of the multimodal facial images of 52 people (14 females, 38 males) obtained by Kinect. The data is captured in two sessions happened at different time period (about half month). In each session, the dataset provides the facial images of each person in 9 states of different facial expressions, different lighting and occlusion conditions: neutral, smile, open mouth, left profile, right profile, occlusion eyes, occlusion mouth, occlusion paper and light on [Figure 1]. All the images are provided in three sources of information: the RGB color image, the depth map (provided in two forms of the bitmap depth image and the text file containing the original depth levels sensed by Kinect) as well as 3D. In addition, the dataset comes with the manual landmarks of 6 positions in the face: left eye, right eye, the tip of nose, left side of mouth, right side of mouth and the chin. Other information of the person such as gender, year of birth, glasses (this person wears the glasses or not), capture time of each session are also available.
 
More information about the database can be found here.

Link: http://rgb-d.eurecom.fr/
32. The MUG Facial Expression (2010)

The database consists of image sequences of 86 subjects performing facial expressions. The subjects were sitting in a chair in front of one camera. The background was a blue screen. Two light sources of 300W each, mounted on stands at a height of 130cm approximately were used. On each stand one umbrella was fixed in order to diffuse light and avoid shadows. The camera was able to capture images at a rate of 19 frames per second. Each image was saved with a jpg format, 896×896 pixels and a size ranging from 240 to 340 KB.

In the database participated 35 women and 51 men all of Caucasian origin between 20 and 35 years of age. Men are with or without bears. The subjects are not wearing glasses except for 7 subjects in the second part of the database. There are no occlusions except for a few hair falling on the face.

More information about the database can be found here.

Link: http://mug.ee.auth.gr/fed/ 
33. The UNBC-McMaster Shoulder Pain Expression Archive (2011)

The database contains: 

1) 200 video sequences containing spontaneous facial expressions, 2) 48,398 FACS coded frames, 3) associated pain frame-by-frame scores and sequence-level self-report and observer measures, and 4) 66-point AAM landmarks. This paper documents this data distribution in addition to describing baseline results of our AAM/SVM system.

More information about the database can be found here.

34. The Yale Face Database (1997)

The Yale Face Database (size 6.4MB) contains 165 grayscale images in GIF format of 15 individuals. There are 11 images per subject, one per different facial expression or configuration: center-light, w/glasses, happy, left-light, w/no glasses, normal, right-light, sad, sleepy, surprised, and wink.

More information about the database can be found here.
See a video about  Real-Time Facial Expression Using Raspberry Pi athttps://www.youtube.com/watch?v=FsdQwKeY5lg
Facial Expression Public Databases
Published: