DCD602 Deceit Detection (II) PDF

DCD602. DECEIT DETECTION (II) 1 Published by Evidentia University of Behavioral & Forensic Sciences 111 E Monument Ave Kissimmee, Florida, USA © 2022 Evidentia University of Behavioral & Forensic Sciences 2 1. Signs of deception....

DCD602. DECEIT DETECTION (II) 1 Published by Evidentia University of Behavioral & Forensic Sciences 111 E Monument Ave Kissimmee, Florida, USA © 2022 Evidentia University of Behavioral & Forensic Sciences 2 1. Signs of deception. Over the years, various studies have highlighted different indicators of deception (Klaver, Lee, & Hart, 2007; Vrij, Edward, Roberts, & Bull, 2000; Vrij, Mann, Leal, & Granhag, 2010), but few have received clear research support. The reality is that it is not possible to provide a comprehensive and clear list of irrefutable behavioral indicators of lying. As we have already warned from the beginning of the chapter, to detect deception all factors must be interpreted together, considering in turn the context in which it occurs (DePaulo, Stone & Lassiter, 1985; Ekman, 1985; Ekman & Friesen, 2015; Vrij, 1998a, 2000; Alonso, 2009) and looking for the relationship between deception and certain psychological processes, which can manifest themselves in those observable behaviors. Covering this need for integration that allows us to detect and demonstrate deception is the main contribution of SAVE (System for Analysis of Validity in Evaluation). The following is a summary of the most significant studies in which the behavioral signs that are related to lying and have scientific support have been proven. However, before begin- ning with the categorization of the various nonverbal expressive channels and their relationship with deception, it is worth highlighting the multidisciplinary approach to nonverbal behavior, addressed by psychology, psychiatry, anthropology, sociology, ethology and linguistics. Expressive channels: Gestures. Poyatos (1994a), in order to analyze the communicative process between human beings, organized the different concepts that existed up to that time and structured them into verbal lan- guage, paralanguage and kinesics. The latter term, kinesics, from the Greek kinesis (movement), refers to the science that studies body movements and gestures. 3  Different researchers of nonverbal communication (Efron, 1941; Ekman & Friesen, 1972; McNeill 1992) identified a multitude of subcategories of hand gestures, which are usually ignored in the detection of deception when assessed as a whole, but which allow discriminating liars from truth-tellers if these subcategories are taken into account (Caso, Maricchiolo, Bonaiuto, Vrij & Mann, 2006). Hands can perform conscious gestures (voluntary and autonomous) and unconscious ges- tures. Taking into account the research carried out so far, we will classify these gestures according to their purpose: those related to a verbal message (illustrators that participate in its exposition and emblems that substitute it) and those related to an emotion (modulators that allow to adapt, exag- gerate, manipulate or neutralize it, as well as to fake it to evaluate or deceive other individuals). Emblems are gestures with their own meaning that most people of the same culture under- stand (Kaulfers, 1931; Green, 1968; Saitz & Cervenka, 1972; Johnson, Ekman & Friesen, 1975; Poyatos, 1977), which are used consciously and voluntarily to send messages to recipients who are close to them and which can be defensive, offensive, informative, etc. When an individual manifests an emblem or part of it, unconsciously and involuntarily, it is known as an emblematic slip and is the result of the presence of a strong emotion (Ekman & Friesen, 1972). Its meaning is the same as that of the emblem itself and can involve any part of the body, although usually the hands, head, facial expressions or posture are used. For example, the famous middle finger or the horns are two of the most universal emblems that are used to insult and that frequently appear in the form of an emblematic slip in certain in- dividuals who express them unconsciously due to the emotion they experience and of which they are aware. Likewise, the gesture of nodding the head when verbally denying or denying with the head when affirming with the voice. According to Ekman, it can be determined that a body movement is an emblematic slip and not an emblem if it is performed partially, that is, if only a part of the emblem is executed. For example, shrugging the shoulders usually manifests itself as an emblematic slip in an ephemeral way with the classic shoulder stretch, i.e., meaning only one goes up while the individual claims to be sure of what he is saying. In the same way, Ekman found that these emblematic slips usually remain outside the main visual field of the person who executes them. 4  As for the illustrators, they are easily understood gestures and linked to the idiosyncrasy of each individual, although by themselves they lack their own message, accompany and reinforce a verbalization (Ekman & Friesen, 1969) and depend on the character, subject and context, it is known that they appear less frequently when lying (Zuckerman et al., 1981; Vrij, 2000; Sporer & Schwandt, 2006, 2007). However, in individuals with lying skills and others who have come to believe their own lies, they may increase. As a general rule, a subject who illustrates his statement a lot, in a congruent way and in step with the verbal message, does not usually lie, conveying a message charged with emotion. On the contrary, when an individual lies, is tired, bored, treats a subject with caution or is indifferent to it, the illustrative gestures diminish. Whoever intends to lie, has to combine his false message with emotions and these with false illustrators. In this case, faced with insufficient cognitive re- sources, the liar discards the illustrators in favor of the false message and emotions (Ekman & Friesen, 1972; Zuckerman et al., 1981). Regarding the relationship between illustrator gestures and deception, a study by Caso et al. (2006) showed that truth-telling individuals used more pointer-illustrators (used to point and/ or accuse, threaten, indicate, or be on the same page with someone) to refer to an object, whereas liars used more ideographic gestures (used to describe abstract ideas), such as a clenched fist to express force (McNeill, 1992). Illustrator gestures should be consistent with the verbal and nonverbal communication of the subject using them and should be symmetrical. When these gestures are not coordinated with the rest of the body or are manifested asymmetrically (e.g., an individual pointing his finger in the opposite direction to where his body and gaze are directed) we can suspect that the gestures do not match their speech, and therefore indicate that they may be lying. Related to emotions are modulating gestures, which are voluntary (conscious) and autono- mous (unconscious) movements of some parts of the body that seek to modify the gesture or facial expression that provokes an emotion in order to adjust it (exaggerate it, attenuate it or adapt it) to what we want to transmit at that moment, but which come to light because they are not harmo- nized with the real emotion. 5  Considering the different types of modulators and applying criteria of simplicity and prac- ticality, we will classify them into: neutralizers, adapters, manipulators and tricksters. Neutralizers are gestures and movements that are the opposite of the gesture that provokes an emotion that one wishes to conceal (Ekman & Friesen, 2003). They can be conscious or uncon- scious, voluntary or autonomous, but they always seek to prevent the gesture or facial pressure that provokes an emotion from being present. For example, when an individual is angry, the effect of this emotion on the oral (usually by compressing the lips) and ocular (usually with tension in the eyelids) areas can be neutralized by smiling and lowering the upper eyelids. However, the ten- sion in the muscles surrounding the mouth and its combination with the lowering of the eyelids, show the falsity of this happiness. Adapters are slight, conscious, involuntary repetitive movements that an individual dis- plays to counteract discomfort or nervousness experienced and to mitigate the tension provoked by certain people, issues and/or contexts (Vrij, & Mann, 2006; Koller, Wetter & Hofer, 2015a; Burgoon, Schuetzler & Wilson, 2015). For example, moving hair away from the face, adjusting the knot of the tie, playing with a pen or other object, are some of these movements. According to the model by Zuckerman et al. (1981), these behaviors known as adaptive behaviors, such as scratching or fidgeting with an object (Ekman & Friesen, 1969) are possible behavioral indicators of deception. However, no relevant relationship with lying has been found (DePaulo et al, 2003; Sporer & Swchandt, 2007), except for an increase in the use of object adapt- ers when lying and a decrease in proprioceptive ones (Koller et al., 2015a). Self-manipulators, as Vrij (2000) called them, seek to mask a microexpression (unexpect- ed, autonomous and fleeting). Although they arise from the unconscious, once they occur, the individual is aware of their presence. For example, when someone has said something that he does not want or should not say, because it is offensive, he quickly and automatically (microex- pression) moves his hand to his mouth to cover it. The manipulator masks the microexpression by reconverting it into a compatible and habitual gesture, for example, touching the hair, the ear or the nose, scratching the face, the eye, the head or the neck. 6  So, is someone touching their ear or scratching their head always using a manipulator to mask a microexpression? No, and we will never be able to say for sure. But if he does it at the very instant he claims he didn’t steal the bike, it’s more likely to be a microexpression than a sudden itch. Paralanguage Prosody or paralanguage is defined as the component of language that refers to the cogni- tive processing necessary to understand and express communicative intentions through supraseg- mental aspects of speech, such as pauses, intonation variations and modulations of vocal intensity (Joanette et al., 2008). Darwin (1872) already described how emotions affect the voice. Fernández- Abascal, Jiménez and Martín (2003) point out that emotional prosody (Monrad-Krohn, 1947) is related to the communication of emotions and that, through it, emotional content is introduced in the message. Today we know that the tone of voice informs us of the emotional state of an individual (Glass, 2003), for example, it is deeper when the topic depresses or saddens him. Also, that the presence of emotions of anger, fear and/or joy increase the speed of speech (Banse & Scherer, 1996) and fear, excitement or anger increase volume. An individual’s high average pitch value (fundamental frequency at which the vocal cords vibrate) reflects a higher degree of arousal, a wide range between its maximum and minimum value indicates exaltation and, while abrupt fluc- tuations between high and low values reflect negative emotions such as fear and anger, smooth ones correspond to positive ones such as joy (Ortego, 2009). We also know that, if the speed of speech increases, the speaker will generally be excited or that if he is euphoric he will speak faster and with fewer pauses, as opposed to if he is depressed. A study conducted by Klaver et al. (2007) found that people with psychopathic traits show typical patterns of nonverbal behavior when communicating, such as less smiling and faster speech. But what indicators are related to deception? 7  For Vrij (2000) the longer duration of pauses when lying, the higher pitched tone of voice and the dilated response latency (Sheridan & Flowers, 2010), an attempt to gain time, are the only objective indicators related to deception (Vrij, 2008). In addition, stammering would appear in individuals who lie, because they feel insecure and hesitant in their words and also in their voice (DePaulo et al., 2003). In conclusion, we can say that there are different speech parameters such as pitch, volume, speed, fundamental frequency and voice quality that allow us to identify emotions in people. These emotions, in turn, allow us to detect consistencies and inconsistencies between what an individual expresses verbally and nonverbally. The longer duration of pauses, the higher pitched tone of voice, the presence of stammering or a dilated response latency, could be directly related to lying. Oculesics From the point of view of nonverbal communication, the eyes are, within the face, the elements that provide the greatest volume of information, regulating or adapting the quality of a conversation. They carry out conscious (involuntary and voluntary) and autonomous commands linked to emotions. They facilitate effective communication with other individuals from the be- ginning and are the first thing we look at when initiating a conversation (Argyle, 1971; Cary, 1978). Gaze is possibly the most important and information-rich channel of nonverbal communi- cation (Kleinke, 1986). The multiple pupil movements, easily analyzable in those whose alignment is regular, but difficult to interpret, are often the subject of simplification. Current scientific research (Porter & ten Brinke, 2010) concludes that there is no eye movement pattern associated with deception, con- trary to the traditional assertions of NLP (Neuro Linguistic Programming) advocates who linked gaze direction to truth and lies. The mistake of stating so lightly and simplistically that a person who looks to the right, while claiming to love the food we have prepared, is lying, for example, is using a tool of great value and complexity in the wrong way. The range of possibilities linked to this eye movement is so wide that any person could move his eyes in any direction without show- ing the truth or manifesting a lie. 8  Wiseman et al. (2012) concluded that no evidence was found to support this idea and re- vealed that there is no relationship between lying and eye movement. Also very common is the myth of the fixed gaze, which spreads the belief that the individual who maintains eye contact with his interlocutor cannot be lying and that, on the contrary, the one who averts his gaze is trying to deceive. Contrary to popular belief, anyone with some skill at deception needs to stare into the eyes of his interlocutor to monitor him, check the effect his lie is having on him (Buller & Bur- goon, 1996) and in turn reinforce it (Kleinke, 1986). In fact, several studies have concluded that individuals who lie maintain more deliberate eye contact than those who tell the truth (Mann et al., 2012; Frosina et al., 2018). According to one of Ex-line’s (1963) investigations, people’s eye contact is affected by the need to hide information. When an individual tries to hide something that generates a strong emotion, he often tries to turn his neck and head away from the stimulus that provokes him, while closing his eyes with his eyelids and sometimes with the orbicularis oculi of his eyes, in order not to process it. These gestures are often combined with fixed glances at the eyes of the interlocutor to check whether the message has reached them. Avoiding eye contact is commonly considered an indication of deception without prior consideration of the character, subject, and context. For example, during interrogation the investi- gator is advised to get physically close to the suspect (Inbau, Reid, Buckley & Jayne, 2013). Such behavior may provoke gaze aversion in the interrogated, as people generally begin to avert their gaze when their personal space is invaded (Argyle & Dean, 1965). If the investigator does not take into account in this case the effect on gaze caused by the intrusion into the respondent’s personal space, it could generate unfounded suspicion. As for the eyelids, their functions include protecting the eye from foreign bodies or unpre- dictable light changes, moistening the eye and promoting sleep by interrupting the eye’s infor- mation processing. But, in addition, the eyelids fulfill a primary function in relation to nonverbal communication, that of regulating incoming visual information flows to facilitate information processing (Hacker, Kuhlman, Kircher, Cook & Woltz, 2014). 9  That is, each time the eyelids close they allow the brain to separate the information gath- ered by the eyes in the form of informational packets. The next time you watch the news, notice how the anchorman reads the news on a screen, and because of this, he keeps blinking. A study by Siegle, Ichikawa and Steinhauer (2008) showed that blinking becomes more pronounced just before maximum cognitive load or maximum information processing, is inhibit- ed as pupillary dilation increases with maximum performance (the pupil dilates with eyelid clo- sure and contracts with eyelid opening), and becomes more pronounced again after that maximum cognitive load. According to this and taking into account that lying is cognitively more demanding than telling the truth (Vrij, Mann, Kristen & Fisher, 2007; Vrij, 2008; Vrij et al., 2009; Walczyk, Mahoney, Doverspike & Griffith- Ross, 2009), it is inferred that lying causes slight oculoskeletal changes, such as pupillary dilation and decreased blink rate, on the grounds of cognitive load when doing so. An experiment by Sharon Leal and Aldert Vrij (2008) showed that liars decreased blinking during deception (i.e., in the target period) compared to their basal pattern, increasing it during the compensation period, once the lie was expressed (compared to the target period and their basal pattern). But these oculesic changes do not only occur on the occasion of cognitive load, they also occur when anxiety increases. Frosina et al. (2018) demonstrated that as anxiety increases, blink frequency increases. Furthermore, these authors observed that liars blinked more from the begin- ning of the interview than those individuals who told the truth. In the same study, they also corrob- orated the research of Leal and Vrij (2008) by stating that decreased blink frequency is associated with increased cognitive load. Other expressive channels. The following is an abbreviated description of other nonverbal expressive channels that play a subsidiary role in the field of deception detection research compared to other channels, such as facial expression or paralinguistics. Perhaps the difficulties involved in their scientific treatment (López, Gordillo León & Grau, 2016) have been the cause of the lack of interest in them and the relationship they have with lying. 10 Positions When someone lies, they provide behavioral clues associated with deception, but they also reveal other clues indirectly compatible with lying that do not in themselves make it evident, but when taken together allow the interpretation and elaboration of hypotheses about the credibility of a subject’s discourse. Ignoring these indications compatible with lying, such as the presence of emotional inconsistencies, would leave us with little choice in the face of someone who is trying to deceive us. Therefore, it is essential to highlight the importance of these clues, since those sci- entifically supported that are directly associated with lying may not be present or, if they are, they may be scarce and/or difficult to observe. The work done by Mehrabian (1972) is the basis of reference on learned body movements, which vary from one culture to another, in the context of liking and disliking another person. Knowing whether an individual dislikes his or her interlocutor in a given context or on a given issue can provide cues consistent with deception, although they are not always triggered by decep- tion. Mehrabian’s research showed that when an individual feels at ease with another person, he or she leans forward more than when experiencing displeasure, the orientation of the body is more frontal and the arms and body are more open. Thus, a lying subject, upset at feeling cornered, may lean backwards more frequently, his body may be oriented in a different direction from that of his interlocutor, and his arms and body may adopt a more hermetic posture. By analogy with emblematic slips, described in the gestural expressive channel, it is im- portant to mention corporal slips, which are involuntary gestures of some part of the body that reveal a hidden intention. For example, taking a step backward for no apparent reason, but driven by an emotion that manifests the need to separate from something or someone that generates rejec- tion or antipathy. Also self-shrinking or slouching are slips that denote shame that can be caused by the lie itself. A study by Frosina et al. (2018) showed that those individuals who lie make more changes in trunk movements than those who tell the truth, and these changes are more significant when the liar narrates his or her story in reverse order in the context of a cognitive interview. 11  But if something generically characterizes liars, it is the fear of being detected and having to assume the consequences of their lie, which generates reactive defense responses referred to as FFF (“Fight, Flight, Freeze”), which protect from dangers and threats through fight, flight or immobilization (Crosby, 2008) and involve the activation of the motor and excitation centers of the brain. In the presence of a danger such as the possibility of being betrayed, the liar tends to freeze (paralysis while the brain analyzes the risks and possible actions). If he decides that fighting or fleeing are viable options, he activates certain systems that allow him to refute his argument or abandon the conversation and leave. But if it mentally concludes that there is no way to emerge victorious, motor immobility patterns are activated, resulting in tonic immobility, i.e. body paral- ysis. This paralysis, in animals, is considered an evolutionary reaction of adaptation when it is not possible to resist the attack of a predator. Proxemics. Knapp (1982) defined proxemics as “the study of the use and perception of social and per- sonal space”, without mentioning that it is a mostly unconscious and cultural process. Hall (1968) classified this personal space in turn into: intimate space, causal-personal space, social-consulta- tive space, and public space. With regard to the application of this expressive channel in the field of deception detec- tion, there are no proxemic indicators related to lying. We can only affirm that whoever invades without justification the intimate or personal space of another individual causes discomfort and is perceived as a threat or aggression, preparing the body for the fight. Haptics is the expressive channel that encompasses what concerns tactile perception, i.e. touch. It leads us to emotion (Gallace & Spence, 2010; Hertenstein, Verkamp, Kerestes & Holmes, 2006) and is a source of stimulation and development. Although this expressive channel may be the cause of misunderstandings and mispercep- tions between people belonging to contact and non-contact cultures (Hall, 1963), there are no behavioral cues related to haptics that are useful for deception detection. 12 2. Facial Expression Analysis Protocol (FEAP) Next we present the facial expression analysis protocol, which for international projection we have decided to call FEAP (Facial Expression Analysis Protocol). It has its theoretical basis in three discoveries of great importance in modern psychology: the universality of facial expression of basic emotions, the existence of differentiated neural pathways for intentional and automatic facial movements, and the cognitive thematic nuclei associated with emotions. What it is and what it is not. Before going into the development of the protocol, we would like to dedicate a few lines to the contextualization of this tool within the field of deception detection. We have to say clearly that this protocol is not a tool to detect deception, it is a tool to detect consistency or inconsistency in emotional facial expression. This in many cases can be linked to an intention to deceive, but we must assume that there is not enough scientific evidence to assess in which cases it will be assimilated to deception and in which cases it will not. For this reason, this protocol will be of great interest to the analyst but, being conservative in our conclusions, we must consider that the detection of an inconsistency will be a red flag and not an indicator of lying. This alarm signal will guide the investigator in the search for evidence of the truth or falsehood underlying the testimony of the subject under analysis. (López, Soto, Gordillo, & Pérez-Nieto, 2014). Porter and ten Brinke (2008) raise the point of false positives that can occur when using facial expression analysis as a deception detection tool. The presence of inconsistent facial ex- pressions in certain genuine expressions can lead to the misinterpretation that emotion is being simulated (false positive). However, the authors are clear that “empirical documentation of in- consistent emotional expressions in simulated facial expressions offers exciting new possibilities for advancing the scientific understanding of human emotion and, we hope, will lead to needed reforms in training within the forensic and security domains.” 13  Figure 1 Protocol development The FEAP protocol is structured along 4 consecutive phases of very simple elaboration (see figure 2). It is applicable to the analysis of one or several areas in the search for emotional consistency. The protocol is developed as follows: - Phase 1. Expected Emotion (E.E.) In this phase, applying what was suggested in the previous pages, we will analyze the the- matic nuclei of the emotion in the search for the emotion that supposedly should be elicited in the given situation being experienced by the subject(s) under analysis. For example, according to the thematic nuclei, faced with the loss of a loved one, the prototypical emotion that should appear would be sadness and, linked to it, its facial expression. Anger (in the form of outrage) would ap- pear when facing an offense or false accusation. Achievement or accomplishment, joy. And so in each of the situations, according to the cognitive interpretation of this, we will raise the E.E. - Phase 2. Displayed Emotion (D.E.) In this phase we will have to determine which facial expression the subject in question shows on his face. It is simply a matter of analyzing what we see, what is the emotional expression that appears in the subjects under analysis. 14  A special case will occur when the subject does not express any emotion on the face. This possibility should lead us to analyze the reason for the lack of facial expression. We must evaluate causes related to possible disorders, alexithymia, autism spectrum disorders, psychopathy, as well as causes of physical origin such as facial paralysis or use of botulinum toxin. If there is no D.E., we will not be able to use this protocol. In any case, reflection on this fact, prompted by the attempt to use the protocol, may provide us with vital information. - Phase 3. Visual comparison. Once we have established what there should be (E.E.) and what there really is (D.E.), what we observe in the face of the person under analysis, we can conclude a first level of consisten- cy-inconsistency. We propose in this protocol an initial, quick, visual analysis that only needs a few minutes and a brief reflection applying the optics of emotion. We propose this first level of rapid analysis because we have found that in many real-life situations, an inconsistency and, therefore, an alarm signal already appears at this first level. Figure 2. - Phase 4. Detailed analysis. If, in phase 3, we find that there is emotional consistency, we should not yet consider it valid. We will then move on to phase four in which we will perform a detailed analysis, checking the action units present. Using the FACS and EMFACS system, we will analyze in detail the D.E. to verify that the action units present in the face of the person analyzed correspond to the action units that are present in a correctly displayed facial expression, a prototypical expression. 15  If the prototypical facial expression of the E.E. coincides with the facial expression of the D.E., we should understand that there is emotional consistency in the behavior displayed by the subject, otherwise we should posit the hypothesis of emotional simulation. Here are some examples taken from real cases. Situation: The lifeless body of a girl is found. This fact is communicated to the parents. A few hours later, the family home is searched for evidence of the girl’s death. The mother is present at the search. Phase 1. E.E. Given the irreparable loss posed by the situation, the prototypical emotion that should appear will be sadness. Phase 2. D.E. The existing audiovisual material is analyzed and it is concluded that the emotion shown at that moment is one of joy. Phase 3. Quick comparison. There is an inconsistency between E.E. and D.E. that is ob- served without the need to go to phase 4. It will be an alarm signal that should make the investiga- tor consider the origin of this inconsistency. This particular case occurs on many more occasions than we might initially think. It is caused because the subject of analysis tries to befriend the police officers conducting the search. We are not talking about collaborating, we should not con- fuse it with this. A person can be a collaborator and present perfect emotional consistency. We are talking about the presence of a facial expression of joy resulting from an attempt to be liked by the policemen (or by their interlocutors if they are other professionals), even joking with them, when the situation should elicit a very different D.E. The origin of this fact arises from the function of affiliation that the expression of the emotion of joy fulfills in the human being, we unite through joy. Example 2. Situation: A few months old girl goes missing. The mother appears in the media asking the alleged kidnappers to release her. Phase 1. E.E. Given the loss posed by the situation, the prototypical emotion that should appear is sadness. Since hope has not been lost and the child is expected to be returned, this emo- tion may appear and disappear, alternating with other emotions such as anger (usually linked to verbalizations about the captors), but we must understand that when talking about her daughter, the emotion of sadness must appear at some point. 16  Phase 2. D.E. The existing audiovisual material is analyzed and it is observed that the emo- tion shown at that moment is sadness. Phase 3. Quick comparison. EE=DE=sadness. In a first quick analysis, comparing the ex- pected emotion and the emotion shown, we observe that there is emotional consistency and we move on to the next phase. Phase 4. Detailed analysis. We analyzed the action units present in the mother’s face and concluded that when she appears sad, only action unit AU15 (lowering of the corner of the lips) appears. When comparing the action units that are present in the prototypical facial expression of sadness, we see that the units that should be observed are AU15 (lowering of the corner of the lips), AU1 (elevation of the inner part of the eyebrows) and AU4 (lowering and contraction of the eyebrows). In this case we have to raise the hypothesis of emotional inconsistency but particularly in this example, the movement of inner elevation of the eyebrows characteristic of the emotion of sadness is hardly executable intentionally if the emotion is not felt. The real case that inspires this example corresponds to a mother who sold her baby and publicly denounced it as a kidnapping. Later, after seeing the media stir, she returned the money to the buyers and got her daughter back. She presented it to the public as a return by the kidnappers. A few weeks later it was proven that the whole thing was a failed set-up in an attempt to make a profit from the sale of the baby. Example 3. Situation: An athlete is accused of doping. He appears in the public media say- ing that it was all a set-up. Phase 1. E.E. Given the situation, if it really was a set-up, it is to be expected that the person will be outraged by the accusation and at some point the facial expression of anger will appear. Phase 2. D.E. The subject is analyzed and it is observed that the facial expression that ap- pears on his face throughout the public appearance is that of anger. Phase 3. Quick comparison. EE=DE=Anger. In a quick visual analysis, we hypothesize emotional consistency. 17  Phase 4. Detailed analysis. It is found that the action units present in the prototypical emo- tion of anger are AU4 (lowering and contraction of the eyebrows), AU5 (upper eyelid elevation), AU7 (eyelid tightening), AU17 (chin elevation), AU23 (lip thinning), AU24 (lip tightening) and AU38 (nostril dilation). In the detailed analysis of the expression shown, it is concluded that all these action units are present. The conclusion should be of emotional consistency and therefore, the analyst should conclude that the subject really feels outraged. 3. NBAM Protocol We present below the protocol for nonverbal behavior analysis called NBAM (Nonverbal Behavior Analysis Matrix). The incorporation of the term “matrix” is due to the fact that the structure of the protocol resembles a matrix in which different data sources are crossed. Starting from the initial data ma- trix, and after a process that we will explain in the following pages, the conclusions of the analysis will be obtained. The NBAM protocol is based on the existence of certain types of behavior that are associat- ed with the conscious or unconscious perception of a stimulus. In this way we can establish a first classification of types of behavior according to whether it is intentional or unintentional (automat- ic) (Palmero & Mestre, 2004). In addition, the subject can inhibit or control their behavior. Total inhibition of the behavior will not be observable and neither will its control, if it is well executed. However, there are situations in which an automatic behavior is triggered that the subject tries to control it, but the attempt is unsuccessful, resulting in behavioral slips that make it possible to observe this attempt at control. This specific case of failed control attempt is of special interest for the analyst since it assumes that the subject intends to hide this particular behavior. 18 For this reason we include it as a special case, deriving from the above that the elements related to the behavior that we can observe in an analysis will be: - Intentional conduct. - Automatic behavior. - Failed behavioral control. - Absence of conduct. Figure 2 shows the reader an adaptation of the scheme developed by Palmero and Mestre (2004), which shows the emotional process, starting with the appearance of a stimulus and leading to a certain behavior. In the graph presented, the different concepts are represented in three colors: red, blue and green. In this first diagram we present the complete graph (conscious and non-conscious percep- tion). Since it does not affect the interest of the present chapter, for educational purposes, we will henceforth refer to the process by eliminating the non-conscious perception (green squares). In this way, we simplify the scheme and it does not affect the specific objective of this chapter. Figure 2. Emotional process of conscious and non-conscious perception of a stimulus presented by Palmero and Mestre (Palmero & Mestre, 2004). 19  In red we have processes that are not observable and that at all times, our conclusions about them will be inferences. For example, we can observe a certain reactive facial expression resulting from the emotion of sadness. In this case, what is observed will be a certain automatic facial expression. Saying that this expression means that the subject is sad is an inference, because sadness itself is not observable, only its expression is observable. The basis of this tool is precisely this, to clearly understand that the only observable thing is the behavior or its absence (intentional, automatic or failed attempt at control). Behavior is that objective part, although not 100% so. Blue shows what can be observed. As a result of the subject’s motivation and cognitive processes, a certain intentional behavior may be displayed. As a result of the emotional experi- ence and their own evaluation, an automatic (uncontrolled) behavior may appear. As a result of the subject’s emotional experience and cognitive processes, an attempt to control behavior may ultimately be unsuccessful. In addition, a certain physiological response may be perceptible, which may also be ob- servable in some cases, for example, when the person begins to sweat and this sweating is percep- tible by sight. It can be seen that in the diagram, the stimulus is marked in both red and blue. This is be- cause there may be situations in which the stimulus is observable and others in which it is not. For example, in the course of an interrogation, when faced with an engaging question, the subject may display a certain behavior (turning their body towards the door, a certain facial expression, etc.). In this case, the investigator can verify that the stimulus that triggered this behavior was the question, the stimulus is present and observable. In other cases, we can observe a behavior result- ing from a stimulus that is no longer present or that is internal to the subject (e.g. a spontaneous memory). As a summary, it is important to remember that the only observable aspect is the behavior: automatic, intentional and/or the failed attempt to control the behavior; some aspects, if evident, of the physiological response; and, in some cases, the stimulus. As shown in Figure 3, this estab- lishes what we will later see as our first two levels of analysis: the level of observable data and the level of inferences. 20  Figure 3. Level of observable data, and level of inferences. Let’s take a quick look at the elements that appear, from the initial perception of the stim- ulus to the development of a certain behavior. Perception can occur in two ways: consciously and unconsciously. We can allude to those referred to by Palmero and Mestre (Palmero & Mestre, 2004) according to which, in the first case, the subject will notice the presence of a stimulus that captures their attention. Conscious percep- tion will be influenced by a series of filters that modulate perception: cognitive variables, such as beliefs, judgments, etc., that the subject possesses about the stimulus in question; biological con- ditioning factors (including the subject’s own personality, discussed in chapter 8) and the subject’s current affective state at that moment. On the other hand, the non-conscious perception of the stimulus will not capture the con- scious attention of the subject, but some processing of the stimulation will occur. This may lead to the next steps of the emotional process. It may also happen that the stimulus demands an im- mediate response, in which case the subsequent steps of the emotional process can be activated. Unconscious perception is also influenced by the above-mentioned filters. 21  Once the stimulus is perceived with sufficient strength, the subject will evaluate and assess it consciously. It is the step prior to the experience of an emotion that will determine the emotion experienced. Evaluation refers to the interpretation and assessment refers to the estimation of the personal impact. Simplifying a lot, we can say that evaluation involves measuring different parameters of the stimulus and assessing is about determining how the situation affects my objec- tives in terms of benefit or harm to the subject, in terms of specificity of the benefit or harm and in terms of subjective probability of controlling the situation. The aforementioned filters also have an appreciable influence on the evaluation-assess- ment, biasing and individualizing this process. As a consequence of this process, the subjective experience of emotion (thrill, feeling), a physiological response, a specific motivation with respect to the perceived stimulus will be trig- gered. Regarding the concept of emotion, which has been extensively discussed in previous chap- ters, we only present below the definition provided by Fernández-Abascal, Jiménez and Martín (2003). “Emotion is the set of effects produced by a multidimensional process, responsible for analyzing particularly significant situations, interpreting them subjectively according to personal history, expressing them to others, preparing for action and generating changes in physiological activity”. We recommend the reader to read this definition carefully and internalize it as it will be an essential foundation for the analysis of a subject’s behavior. On the other hand, as we saw in chapter 2, the concept of cognition refers to the processing of information based on perception and knowledge, according to which the environment is rep- resented, information is received and acted upon”. Finally, following the presented graph we have to briefly explore the concept of motiva- tion. Motivation should refer to an adaptive process, which is the result of an internal state of an organism, which drives and directs it towards an action in a certain direction. That is, it is influ- enced by external and internal factors that activate the organism and direct its behavior towards the achievement of rewarding objectives or goals (Fernández-Abascal et al., 2003). The concepts of energy, direction and maintenance are involved. 22 Baseline. The fact that perception, evaluation and assessment are influenced by personal filters makes it imperative that we take into account the individual differences that are incorporated into the typ- ical perception-behavior process. As reflected by the orange lines in Figure 4, the aforementioned filters (biological dispositions, judgments, beliefs, and affective state) will therefore indirectly affect behavior. Figure 4. Indiidualization of behavior. This implies the need to establish a baseline behavioral pattern, a baseline, on which we can establish behavioral changes that are interesting for the purpose of the analysis. For exam- ple, in the case of analyzing an interrogation or an interview, it will be interesting to analyze the individuality of each subject by establishing a baseline. In this way we will be able to compare behavior at different moments in time, allowing us to observe changes that raise questions such as: Why has your behavior changed? What emotional, motivational or cognitive processes have triggered the change? Is the stimulus causing the behavioral change internal or is it due to a reaction to a stimulus present in the environment? Is the stimulus that triggered the change one of the interrogator’s questions? 23  Ideally, the baseline behavioral pattern should be established by analyzing audiovisual ma- terial of the subject at a different moment in time, ideally at several different times. In this way it will be established how the individual behaves in situations different from the one analyzed, and the origin of the behavioral changes can be inferred. Given that this is difficult in most cases, an appropriate solution in the case of interrogations will be to establish a banal conversation with the subject to be analyzed in which we observe his/her behavior and compare it with the situation in which there is a high emotional involvement. The existence of these filters and the fact that they affect behavior implies that inferences can also be made about them through behavior. As we saw in Chapter 8, personality filters can be inferred through mere observation. Expressive channels. Another fundamental element of the protocol is the establishment of a series of nonverbal expressive channels that are analyzed by extracting behavioral data. Thus, the expressive channels are: facial expression (FE), gestures (G), postures, orientation and movement (P), paralanguage (PA) or emotional prosody, proxemics (PR), haptics (H), appearance (A), oculesics (O), and the necessary verbal channel (V) that will be used to perform linguistic content analysis and to es- tablish the synchrony between verbal and nonverbal behavior (Figure 5). All of them have been analyzed in detail in the preceding chapters. Up to this point, the three basic elements that form the theoretical basis of the protocol have been presented, from which it follows that: a) The only observable aspects are behavior (automatic, intentional or failed attempt at con- trol), some physiological responses and, in some cases, the stimulus. b) From these behaviors, the analyst must infer the emotional, cognitive and motivational states of the subject under analysis. c) That there are filters (biological, states of mind, values and beliefs) that individualize be- havior, and therefore it is necessary to establish a base pattern of behavior. d) That it will be necessary to perform the analysis through each of the expressive channels. 24  Figure 5. Individualization of behavior. Procedure-tables. Having reached the practical point of applying the protocol, the first thing to establish is the different levels of analysis. The NBAM tool is structured along five levels numbered from level 0 to level 4 which carry out the following process: - Level 0. Base pattern. - Level 1. Observable behavioral data analysis - Level 2. Drawing inferences based on observed data. - Level 3. Establishment of working hypotheses - Level 4. Last level in which there is a call to action. This call could well be the establish- ment of proposals for action, proposals for improvement, etc. and this will depend on the profes- sional field in which the protocol is being used (see figure 6). 25  Figure 6. NBAM protocol process. Now let us take a look at how to carry out each of the proposed levels. The reader may find it difficult to understand, just by reading the following lines, the practical implementation of the protocol. This is logical, but we are convinced that, after viewing a practical case study such as the one presented in the next chapter, these doubts will be cleared up. 5.1. Level 0. Baseline. At this level we must try by all means to establish the way in which the subject under analysis behaves in normal situations. When this is impossible, as we have pointed out, we will establish the pattern by asking control questions. We must establish base patterns in each of the expressive channels, so that the behavioral particularities of the subject in each of them will be collected. Each data point will be coded including the letter B (for baseline), followed by the channel code and ending with the observation number, so that, for example, baseline observations on Fa- cial Expression will be coded as: B-FE-1 to B-FE-n, where B corresponds to baseline, FE to facial expression and n to the number of the last observation of that channel (the observations will be numbered consecutively starting with number 1). 26  Each observation will be collected in a row of the table, adding as many as necessary. Like- wise, if there is no observation for any of the channels, it may be omitted. In other words, rows with blank channels for which there is no behavioral observation will not be included. Level 1. Data. At this level we will collect, in each expressive channel, each of the behaviors that we consider of interest. As many rows will be added for each channel as necessary. That is, we can have several rows for facial expression data, several rows for gestures, and so on. Each data will be numbered as follows: Each data will be coded by including the letter D (data), followed by the channel code and ending with the observation number, so that, for example, observations of Facial Expression data will be coded as: D-FE-1 to D-FE-n, where D corresponds to data, FE to facial expression and n to the num- ber of the last observation. 27  This level is of utmost importance as it is the basis of the protocol. If this level is done correctly, in an objective manner, without including our opinion about what was observed, it can serve as an objective basis to share with several analysts. If several analysts arrive at the same hypotheses, these will be significant and should be taken into account. Each piece of data will specify the frame associated with the behavior, the verbalization that the subject was making at the time of performing the specific behavior and a description of the behavior. Figure 8. Data level table. 5.3. Level 2. Inferences. At this level, the analyst will make inferences regarding the emotion, cognition and moti- vation of the subject for the performance of the reflected behavior. At this level all the channels are already linked to inferences, which will be coded with the letter I (inference) followed by a correlative number for each of them. Next to the inference, in parentheses, the data points from which the specific inference is drawn will be displayed, so that, next to it, the data from which the inference originates can be read. Subsequently, the number of data points from which the inference is drawn and the number of different channels will be counted. 28  The number of data points will be multiplied by the number of channels, and the result will be reflected in the corresponding table. This will produce a weighting that will give a higher score to those channels with a lot of data, coming from several different channels. This weighting is based on the fact that an inference about motivation, cognition or emotion, if it comes through several channels simultaneously, is more likely to be reliable. For this reason, a higher reliability will be given to inferences that have obtained a high value. However, it is necessary to bear in mind that this numerical value is an aid for the analyst, who must assess whether there is any in- ference that, although it has a low value, has characteristics that make it of great importance for the establishment of the hypotheses. For example, an inference that a person feels anger will have a higher value if it comes from data from different channels than if it comes from a single channel. When a person intends to feign, he or she will pay attention to one of the channels (or a small num- ber of them), but it will be difficult to do so for all channels. Therefore, in this protocol we assume that an inference that comes from a large number of channels is more reliable. However, a simple truthful facial expression of surprise presented on a person’s face when faced with information that he supposedly knew, could be considered of great importance even if it comes from only one of the expressive channels. For this reason, we reiterate that the numerical score given to the inference is for the ana- lyst’s guidance and therefore, in the case of issuing an expert report, this value will not appear in the table. Figure 9. Table of inferences (analyst version and report version). 29 Level 3. Hypothesis. At this level, the analyst establishes their final hypotheses. All of them must be based on the inferences made. Next to the hypothesis, in parentheses, the codes of the inferences from which that particular hypothesis is drawn will be listed. All possible hypotheses should be collected as long as they are based on previous infer- ences. Figure 10. Hypothesis table. 5.5. Level 4. Intervention proposals / operational suggestions. This is the level that closes the protocol. In it, the analyst makes a call to action. Depend- ing on the field for which the analysis has been performed, this level may establish a proposal for intervention or operational suggestion such as “the subject’s statement should be taken again” if the field of work is a judicial investigation or “the person being analyzed should improve his hand gestures” if the protocol is used to advise a public figure. If the report has been requested for surveillance purposes, level 4 can include the estimated future behavioral patterns of the subject under analysis. 30  Figure 11. Table of intervention proposals. Real-time use. There have been many situations in which audiovisual material was not available for anal- ysis. In environments as particular as intelligence, in which the mere recording of an interaction with other colleagues or informants can be a serious problem, there is no choice but to resort to real-time analysis. We need to perform an “on-the-fly” analysis that allows us to analyze the dif- ferent channels and make the necessary inferences and hypotheses. This will be a really complicated job in which you can only succeed if you have previously trained a team in which one person conducts the verbal interview and another or others analyze the nonverbal interactions. Obviously, the training of this team must be intense, starting with a thorough training in the use of this tool and a subsequent implementation in a real-time training. The team should start with the analysis of audiovisual material, then move on to intensive training in real-time analysis, working with actors who simulate different situations, and finally move on to a phase of implementation in a real environment. The minimum team unit should be a pairing of two trained people, who work together in a stable way and between whom there is a good personal relationship. This is the only reliable way to deal with a real-time analysis. It would even be of interest to create a macro-team consisting of different pairs that pool their experiences in the application of the real-time protocol. 31  As noted above, by combining the theoretical approach of this chapter with the practical application shown in the following chapter, the reader will be able to accurately apply the NBAM protocol. 32