Exploring deception using automatically detected facial action units

Tools

Vinkemeier, Doratha (2020) Exploring deception using automatically detected facial action units. PhD thesis, University of Nottingham.

[thumbnail of Corrected Version] PDF (Corrected Version) (Thesis - as examined) - Repository staff only - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (21MB)

Abstract

Lie detection has always gripped mankind. Today, applications range from individual employee screening to mass terror scenarios. Yet, signs of deception are still not well understood and there is general agreement that humans are bad at detecting them. We suffer from bias and subjectivity as well as a lack of stamina and observational acuity. For this reason, hope is now being placed on automatic methods such as action unit (AU) detectors, which detect facial muscle movements that can reveal affective states.

Automatic AU detectors are still in their developmental stage; they are proving, however, to be useful for both detecting and learning about deception. This thesis used

CNN-BLSTM and OpenFace AU detectors and decision trees in two different deception scenarios. In one of them, the game of poker, deception is integral and desirable. Videos obtained from the University of Southern California showed pairs of players who communicated over a network and behaved spontaneously in a laboratory setting. I ascertained that players who were folding, as opposed to calling or raising, displayed significantly more AU12 and AU5, action units associated with smiling and other emotions, whereby CNN-BLSTM and OpenFace showed only limited overlap.

The study of deceit is hindered also by the lack of relevant datasets that simultaneously have a ground truth. For that reason, the second part of my thesis was dedicated to building and researching such a dataset - the dice rolling experiment - where participants roll a virtual die and decide themselves whether or not to lie to increase their earnings. This dataset consists of over 1.7 million frames of good quality video along with concurrent mouse tracking information and timestamps of events covering 373 different subjects. It has a defined ground truth and also investigates the effects of cold water stress on deceptive behaviour. This experiment revealed that males lied more than females and that stress reduced lying. Low detection levels and distinct patterns of false positive facial AUs lead me to use head pose estimators, which showed that under stress, deceptive participants moved their heads significantly more than honest ones.

In summary, this study automatically detected scenario-specific clues of deception, explored the limitations of current AU detectors, and generated a large, novel data set uniquely suitable for studying deception and its automatic detection.

Item Type: Thesis (University of Nottingham only) (PhD)
Supervisors: Valstar, Michel
Wagner, Christian
Keywords: Computer vision, Affective computing, Action units, Emotion recognition, Deception, Machine learning, Behavioral economics, Poker, Stress.
Subjects: B Philosophy. Psychology. Religion > BF Psychology
Q Science > Q Science (General)
Faculties/Schools: UK Campuses > Faculty of Science > School of Computer Science
Item ID: 63751
Depositing User: Vinkemeier, Doratha
Date Deposited: 07 Jan 2021 15:44
Last Modified: 07 Jan 2021 15:45
URI: https://eprints.nottingham.ac.uk/id/eprint/63751

Actions (Archive Staff Only)

Edit View Edit View