science overview

 

The scientific study of emotions began with the work of Charles Darwin in the 19th century. Later, psychologist Paul Ekman pioneered the research of facial expressions of emotions, principally using images. His work led to the categorisation of facial expressions relating to emotions into discrete archetypes (happy, sad, fear, anger surprise and disgust). It is important to note that facial expressions are generated by the contractions of muscles, which in turn may (or may not) deform the overlying skin. Therefore, changes in the position of facial features occur after muscle activation. Electromyography (EMG) involves using electrodes like little microphones which "listen" for muscle activation 1000 times per second (unlike cameras which sample at 30-60 times per second). EMG is highly sensitive and can even picking up micro-expressions which are not observable. Unlike cameras, EMG can also detect changes in baseline muscle tone and record electrical activity directly . This is in contrast to cameras which rely on the indirect measurement of skin overlying the muscle. Indeed, EMG was used to calibrate camera-based facial expression algorithms (Cohn & Schmidt, 2004).   

Measuring facial expressions and emotional responses using EMG is a fundamental tool for researchers in media, marketing, gaming and psychology.

Virtual reality provides a useful paradigm for measuring behaviour and providing controlled environments. However, the most salient facial information is covered by the headset as illustrated by figure 1. hence a different approach is needed.

Fig. 1. The image on the left shows an eye tracking heatmap of face to face interaction. In virtual reality, this important area is largely under cover. 

Fig. 1. The image on the left shows an eye tracking heatmap of face to face interaction. In virtual reality, this important area is largely under cover. 

According to the basic emotions model, the face exhibits felt happiness, sadness, fear, anger, surprise and disgust. Positive valence is immediately recognisable in a smile, whereas negative valence is indicated by a frown of anger or disgust. A series of studies in the 1980's and 1990's demonstrated that the facial muscle activation which causes a frown (corrugator supercilii) is related to a decrease in valence. Conversely, the facial muscle activation of a smile (zygomaticus major), is correlated with an increase in positive valence.

Fig.2. Diagram showing the relationship between the facial muscles and that indicate positive (zygomaticus major) and negative (corrugator supercilii) valence.

Fig.2. Diagram showing the relationship between the facial muscles and that indicate positive (zygomaticus major) and negative (corrugator supercilii) valence.

Fig.3. Faceteq solution

Fig.3. Faceteq solution


Further Reading

If you are interested in discovering more about our technology we have provided a selection of papers by members of Emteq's team of scientists and advisors, as well as some foundational papers in this field.