A Tool for Expressing Dynamic Emotional Depth in UX Artifacts
Introduction to this Work
During my career as a UX professional, I created, and helped to create many journey maps, personas, empathy maps, etc. Understanding how people are affected when interacting with the things we build is useful for UX efforts, because it helps us figure out the degree to which we are making something better them. Let’s look at journey maps, for instance — including the user’s emotional state is a common practice. Historically, I’ve usually called this line “sentiment,” but I’ve seen it simply called “emotions” and other names.
Sometime a while ago — honestly, I think it was back around 2021 — I started thinking about emotional depth. I ran across the 2DES, or 2-dimensional emotional classification scale. Unfortunately, I don’t remember the specific article that led me to the research, but according to where I located the original research paper1 which documented the use of emoji to represent emotional states, the article has been cited more than 50 times. That research article led me down a path back to what seems to be the seminal work on the topic2 of mapping 2-dimensional emotional states on a 2-axis grid of “arousal” and “valence.” I also found some additional work from 20053, which includes findings on the human central nervous system and how we perceive the levels of arousal and valence with regard to emotional stimuli – interesting stuff.
What struck me about the original 2D emoji chart was that there were holes. All four sections of the grid had emoji to represent the emotional states on the outer perimeter. I noticed there were no emoji plotted on the interior of the grid, but only eight were missing. Since the foundation was there, I believed it was possible to determine the expressions for the remaining emoji. I began working on the emoji over time and eventually arrived at what seemed like a complete grid of 25 emotions. Figure 2 below shows my initial pass at building the remaining 8 emoji. Since I was building the emoji from scratch, I also tweaked a few of the others for additional consistency. Another resource I used was the work of Marc Brackett, Ph.D. of Yale University in his book Permission to Feel4. I used this work to determine some relative names for the emotional states on the grid, which are noted below each emoji. My new emoji are placed on a field of red in the image below.
What We Really Need is an Interactive Tool!
Need might be a bit hyperbolic, but I was convinced I was on the path to a useful tool. From an aesthetic standpoint, I wasn’t satisfied with the work at this time, and believed what I needed to do was to enhance the emoji with a bit more stylistic depth. So, I began the process of designing some 3D-style emoji that had shading and a bit more depth to the expressions. I added some eyebrows, furrowed brows, some teeth, etc. At the same time it occurred to me that it would be nice to be able to interact with the grid somehow — meaning it would be nice to be able to pick a relative value on the arousal and valence scales, and see the resulting emotional state. Figure 3 below shows a few designs from that batch of work.
Top, left to right: thrilled, stressed, surprised, ecstatic, Bottom left to right: despair, tranquil, anxious, nervous.
I completed the full set of 25 emotions, and began work on the interactive calculator. The calculator allowed the user to select a level of arousal (strength of their emotion) and valence (degree of positivity or negativity of their emotion), which dynamically updated the emotional state. It was a neat tool, but as I was testing it out I felt it still lacked some expressive depth and the emoji seemed a bit cartoonish. So, I began working on an enhanced version.
Me and Generative AI Art
Sometime in mid 2022 I began using the earliest version of Dall-E 3. I was amazed that it could create images from scratch based on a text prompt. In the beginning, I had no idea what to create with it, so I made a bunch of wallpapers and some characters for books I might eventually write. These were fun experiments, but they weren’t a serious application of the technology. Then, in November of 2022, Chat-GPT was released, and that’s when I really fell down the rabbit hole with generative AI. I subsequently signed up for several other image generators as well. My favorite of those was and still is MidJourney — the style of outputs from MidJourney are the most subjectively interesting to me, and I seem to be able to get the most consistent results on that platform. That being said, the most recent improvements to Dall-E 3 have made the difference more about preference to me now, as the results from Open AI seem to be on par. Figure 4 below shows two examples of a wallpaper I generated with MidJourney. Both full resolution wallpapers are also available at the links below:
My Journey to a Completed 2DES Calculator
I created, and helped to create many journey maps, personas, empathy maps, etc. … including the user’s emotional state is a common practice.
One thing that struck me was that expressing emotions on this linear scale, as is typical in these artifacts, lacks the depth I discovered in my research into the 2D Emotional Classification Scale. So, I decided to apply my knowledge of generative AI, design, and development to develop a tool that helps express emotions a bit more dynamically. I wanted it to be easy to use and based on real research into how emotional models work. Below you can see four examples of some sentiment lines I found in some instructional posts on how to create Journey Maps. The direct links to those posts are listed below figure 5, if you would like to read those for yourself:
• From top to bottom: Figma, LucidChart, UX Design Institute, Forbes Advisor
- Forbes Advisor:
- UX Design Institute:
- Lucid Chart:
- Figma:
I decided to go back to the well of MidJourney to see if I could generate some fictional characters expressing the emotions on the 2D Emotional Classification Scale grid. I wanted to see if I could get enough variation and accuracy to create the expressiveness I desired in order for the calculator to have more visual impact and emotional resonance with the user. I collected my 3D emoji, as well as a large selection of 3 human faces representing each emotion — totaling four reference images for each emotion. I supplied those images to MidJourney, and generated some illustrated characters expressing the full range of 25 emotions represented on the 3D Emotional Classification Scale grid. Now that I had images I was happy with, I was able to complete the 2-Dimensional Emotional Scale Calculator.
• From left to right on the top: thrilled, stressed, surprised, ecstatic.
• From left to right on the bottom: despair, tranquil, anxious, nervous.
I built the calculator in plain HTML, CSS, and JavaScript — mobile first, and I ran it through the paces of testing. I tested it on my own to ensure the functionality was where I wanted it, and then I conducted some field tests to validate whether or not the results accurately represented the emotional states of the users. I also socialized the calculator with friends and family, as well as a select group of professional colleagues. Currently, the calculator seems to be performing well, and the emotional states to date have been well-received in terms of accuracy. If you would like to test the 2-Dimensional Emotional Scale Calculator, you can try it below (live). Included in the calculator is a link to a survey. If you’d like to help me improve the 2-Dimensional Emotional Scale Calculator, please fill out the survey — I’d love to hear your feedback.
Select the sentiment and intensity levels below to view your user’s 2DES level.
Both selections are required to generate the output below.
Intensity:
How strong or weak are the user’s feelings?
Sentiment:
How positive or negative are the user’s feelings?
An Evolving Effort
I’m pleased with the results of this work, but I have more to do. Eventually I would like to collect this and other tools I am working on into a useful suite of tools for UX professionals — or professionals any other fields — to help with their work. These types of tools are actually quite fun to build, and they fulfill me because I get to contribute to the community. If you have any ideas for interesting tools like this, I’d love to hear about them. Maybe you have some research that could be turned into a useful tool. If so, please reach out to me and let’s chat.
-Stanton M. Brooks II
Citations
- Posner J, Russell JA, Peterson BS. The circumplex model of affect: an integrative approach to affective neuroscience, cognitive development, and psychopathology. Dev Psychopathol. 2005 Summer;17(3):715-34. doi: 10.1017/S0954579405050340. PMID: 16262989; PMCID: PMC2367156.
External link - Russell, James. (1980). A Circumplex Model of Affect. Journal of Personality and Social Psychology. 39. 1161-1178. 10.1037/h0077714.
External link - Posner J, Russell JA, Peterson BS. The circumplex model of affect: an integrative approach to affective neuroscience, cognitive development, and psychopathology. Dev Psychopathol. 2005 Summer;17(3):715-34. doi: 10.1017/S0954579405050340. PMID: 16262989; PMCID: PMC2367156.
External link - Marc Brackett, Ph.D. (2019). Permission to Feel: Unlocking the power of emotions to help our kids, ourselves, and our society thrive. ISBN: 978-1-250-2184-9.
External link
