Idea Transcript
How to develop emotional intelligence tests - GECO - GERT Marcello Mortillaro Katja Schlegel SWISS CENTER FOR AFFECTIVE SCIENCES
Geneva Emotional Competence (GECO) Test Emotional intelligence in the workplace
Marcello Mortillaro Katja Schlegel
SWISS CENTER FOR AFFECTIVE SCIENCES
Take home message Try to use a theoretically informed approach to measure EI that is based on: - Emotion theory - Specific models for each task
SWISS CENTER FOR AFFECTIVE SCIENCES
Two streams of measurement of EI As a trait via self-report questionnaires (trait EI) As a cognitive ability via performancebased tests (ability EI) SWISS CENTER FOR AFFECTIVE SCIENCES
Ability Emotional Intelligence EI is conceived as a cognitive ability that can be measured with performance-based tests, i.e., tasks that have a correct answer. Ability EI is similar to other types of intelligence in the sense that people can be ranked on emotion-related tasks.
SWISS CENTER FOR AFFECTIVE SCIENCES
MSCEIT Problems • Consensus based scores: 1) validity of the criterion; 2) difficult to differentiate the subjects with a really high EI score; 3) creative response style is disadvantaged • Likert scales rather than «pick the right answer» • Not specific to the work setting
SWISS CENTER FOR AFFECTIVE SCIENCES
Features of the new test -
Ability EI Performance based Organized in sub-tasks based on EmotionTheory Builds upon recent advancements (e.g., situational judgment tests – STEU - STEM) - Concrete real(istic) scenarios - Good discrimination of individual with high EI - Response options built according to appropriate theoretical models
SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion interaction process
Expresser
Observer Expresser
Event Appraisal
Perception
Physiology Expression Motivation
Appraisal Subj. Feeling Physiology Motivation Self (monitoring)
Appraisal Observer’s Emotion
Subjective feeling
Emotion Understanding
Expression
Emotion Recognition
Action Expression
Emotion Regulation Emotion Management
GECO Test Emotional Intelligence in the workplace Structure Emotion Recognition Emotion Management Emotion Understanding Emotion Regulation
SWISS CENTER FOR AFFECTIVE SCIENCES
Critical incidents interviews • Interviews with 38 managers & HR employees to identify emotional scenarios at the workplace and actual and possible reactions • Anger - Fear - (Inappropriate) Happiness Sadness • Initial transcription pool (195 items, 3-5 sentences each) Used for Emotion Management, Emotion Understanding, and Emotion Regulation SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Management The ability to respond in an effective way to somebody’s else emotions In this subtest, the respondent chooses the option that describes the behavior that they would most likely act in response to somebody’s else emotional behavior SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Management The items describes interactive real-worklife scenarios in which another person experiences one emotion (anger, fear, sadnes, inappropriate happiness) that needs to be coped with. The participants choose how they would react.
SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Management Response options represent 5 conflict management strategies (Thomas, 1992) Avoidance Competition Collaboration Compromise Accommodation
SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Management Scoring of correct response based on its “adaptiveness” The adaptiveness of the behavior varies according to objective factors (the specific context, availability of resources, etc.) and is defined according to conflict management theory (e.g. urgency of the response, type of relationship, etc.) SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Management The emotions are explicitly stated in the item in order to avoid confusions and overlap with other sub-tests (e.g., emotion understanding)
SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Management Your colleague, with whom you get along very well, tells you that he is getting dismissed and that you will be taking over his projects. While he is telling you the news he starts crying. He is extremely sad and desperate. You have a very important meeting coming up in 10 minutes. What do you do? 1. You take some time to listen to him until you get the impression he calmed down a bit, at risk of being late for your meeting. (compromising) 2. You cancel your meeting, have a coffee with him and tell he has every right to be sad and desperate. You ask if there is anything you can do for him. (accommodation) 3. You tell him that you are sorry, but that your meeting is very important. You say that you may find some time another day this week to talk about it. (avoidance) 4. You suggest that he joins you for the meeting with your supervisor so that you can plan the transfer period together. (collaboration) 5. You cancel your meeting and offer him to start discussing the transfer of his projects immediately. (competing)
SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Management development overview From the transcriptions we generated an initial pool of 75 items First expert check: clarity of the scenarios, and correct correspondence options/strategies (54 items) Second expert check (16 emotion researcher): pick the best Mechanical Turk studies: Different instructions The theoretically defined solution must be the most frequently chosen (minimum 30%) 44 Items (at least 2 per emotion X strategy combinations)
Expert ratings (17 managers): believability score SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Understanding The ability to understand which emotion is felt by somebody in a certain situation In this subtest, the task for the respondent is to choose the label that describes the emotion that most likely is experienced in one scenario.
SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Understanding Items created according to the Component Process Model (Scherer, 2001), an appraisal theory of emotions that predicts a specific pattern of cognitive evaluations (appraisals) for each target emotion. Appraisals include novelty, conduciveness, coping potential, etc. SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Understanding For example key appraisals of anger are • Goal Hindrance /Obstructiveness • External causation • High Coping Ability • Violations of norms
SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Understanding We generated items (n=48) according to appraisal theory for 15 emotions • Expert rating (10 emotion researchers):at least 5 chose the target emotion and no other label was chosen by 3 or more experts. • Mturk study: accuracy used to choose two items per emotion trying to have variance in the « difficulty level» 29 Items:3 for shame, 2 for the other emotions (despair was excluded) SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Understanding Anger
John has to give a presentation in front of many people in a language that he does not speak very well. What is the most likely emotion that John feels?
Irritation
Joy
Contempt
Interest
Disgust
Relief
Fear
Boredom
Anxiety
Sadness Guilt
SWISS CENTER FOR AFFECTIVE SCIENCES
Pride
Shame
Emotion Regulation The ability to use cognitive strategies to modulate one’s own personal emotional experiences
SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Regulation Several researchers reviewed a large variety of possible emotion regulation strategies and provided different taxonomies Strategies vary in term of appropriateness to the situation and have different adaptive values
SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Regulation • Items describe negative emotional realistic situations • The respondent is asked to imagine herself in those situations • The alternatives represent different cognitive emotion regulation strategies that vary in terms of their adaptive value/appropriateness
SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Regulation Participants read scenarios in which they experience negative emotions (fear, sadness, or anger) 4 response options (2 adaptive and 2 maladaptive regulation strategies Choose 2 out of 4 options Adaptive responses are scored as correct Garnefski & Kraaij, 2006 Adaptive Acceptance Positive refocusing Refocusing on planning Positive reappraisal Putting into perspective SWISS CENTER FOR AFFECTIVE SCIENCES
Maladaptive Self-blame Other-blame Ruminating Catastrophizing
Emotion Regulation We generated 44 items from the transcriptions of the interviews and all strategies equally represented within and between emotions • Expert check (emotion researchers): check for the intended strategy in the response options • Mturk study: excluded items with less than 15% score in one of the 3 possible scores - 0, 0.5, 1 31 items SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Regulation You successfully completed a very important project that took a lot of your time. When you return to your daily business, your boss tells you that he is unhappy that you neglected your other projects. You are very irritated about the lack of acknowledgement of your boss. What do you think? 1. You think that you should have been better organized and have worked on all projects at the same time. (self-blame) 2. You think that you have to accept that bosses are never fully satisfied. (acceptance) 3. You think about the very positive feedback by the client for whom you completed the project. (positive refocusing) 4. You think that your boss is always unfair to you and that you should consider quitting if this continues. (catastrophizing)
SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Recognition The ability to recognize which emotion is expressed by somebody on the basis of his/her nonverbal behaviour Adaptation of the Geneva Emotion Recognition Test
SWISS CENTER FOR AFFECTIVE SCIENCES
Emotion Recognition Development of a short version of the Geneva Emotion Recognition Test (GERT, Schlegel et al., 2014)
42 items: 14 emotions, 3 items each
SWISS CENTER FOR AFFECTIVE SCIENCES
Construct validation study N=149 students at the University of Geneva – Situational Test of Emotional Understanding (STEU; MacCann & Roberts, 2008)
– Situational Test of Emotion Management (STEM; MacCann & Roberts, 2008)
– Big Five Inventory (BFI; John, Donahue, & Kentle, 1991) – Cognitive Emotion Regulation Questionnaire (CERQ; Garnefski & Kraaij, 2006)
– Emotion Recognition Index (ERI; Scherer & Scherer, 2011) – Intelligence test NV5R - General Reasoning (Thiébaut & Bidan-Fortier, 2003)
SWISS CENTER FOR AFFECTIVE SCIENCES
Construct validation study Emotion Regulation
Emotion Management
Emotional Understanding
Emotion Recognition
STEM (Emotion management)
ns
.36**
.29**
.28**
STEU (Emotional understanding)
ns
.48**
.56**
.33**
ERI (Emotion recognition)
ns
.35**
.29**
.41**
NVI5-R (general reasoning)
ns
.32**
.29**
.38**
CERQ (adaptive regulation)
.38**
ns
ns
ns
Big Five- Extraversion
.36**
ns
ns
ns
Big Five- Agreeableness
.25**
.16*
ns
ns
ns
ns
ns
ns
-.32**
ns
ns
.23**
.17*
ns
ns
ns
Correlations
Big Five- Conscientiousness Big Five- neuroticism Big Five- openness SWISS CENTER FOR AFFECTIVE SCIENCES
Test Difficulty & Cronbach‘s Alpha M=.45, α=.69
20 items
20 items
M=.66, α=.75
M=.66, α=.59
M=.56, α=.75
42 items
28 items
• Higher difficulty than other ability EI tests • Reliability for emotion understanding is low, but IRT analyses are needed SWISS CENTER FOR AFFECTIVE SCIENCES
Summary • The GECO is a performance-based test to measure 4 emotional competencies in the workplace • It is available online, administration time ~30 min • First evidence for good psychometric quality and construct validity • Next step: Investigation of predictive validity in samples of employees and managers
SWISS CENTER FOR AFFECTIVE SCIENCES
Wrap-up The new generation of EI test should be - Performance based - Informed by emotion theory in its articulation in sub tests - For each sub-test appropriate theoretical framework should be identified - Items should be built as close as possible to real(istic) scenarios using SJT SWISS CENTER FOR AFFECTIVE SCIENCES
Thank you
SWISS CENTER FOR AFFECTIVE SCIENCES