UX Design & Affective Computing

AI has the potential to significantly enhance Human-Machine Interaction (HMI) by incorporating emotional communication. This is often referred to as „Affective Computing,“ where AI systems are designed to understand, interpret, and respond to human emotions.

Potential of AI supported HMI

Emotion Recognition

AI systems can be equipped with the ability to recognize human emotions through various means, such as analyzing text, speech, facial expressions, and even physiological cues. This allows the AI to understand the user’s emotional state and respond appropriately.

Emotion Generation

AI systems can be designed to generate responses that are not only informative but also emotionally resonant. For instance, an AI chatbot can be programmed to respond with empathy and understanding when a user expresses sadness or frustration.

Personalization

By analyzing user behavior and preferences, AI can personalize interactions to create emotionally relevant experiences. This might involve recommending content or products that align with a user’s emotions and interests.

Data Privacy and Security

Analyzing emotions often involves processing sensitive personal data. Ensuring the security and privacy of this data is paramount.

Emotionally Aware Feedback

In various domains, such as education or skill training, AI can provide feedback that takes into account the learner’s emotional state. This can lead to more effective and motivating learning experiences.

Virtual Therapists

AI-powered virtual therapists can help individuals manage their emotions, offering coping strategies and providing an outlet for emotional expression.

Cultural Sensitivity

Emotions are expressed differently across cultures. AI needs to be culturally sensitive and avoid misinterpreting emotions based on cultural nuances.

Healthcare and Well-being

AI can be integrated into healthcare devices to monitor a patient’s emotional state and provide interventions or recommendations. It can also offer emotional support to individuals dealing with stress, anxiety, or depression.

Ethical Considerations

However, there are ethical considerations to be aware of. While AI can simulate emotional understanding, it doesn’t actually experience emotions. Ensuring that users understand they’re interacting with AI, not a sentient being, is crucial to prevent any false sense of emotional connection.

Data Privacy and Security

Analyzing emotions often involves processing sensitive personal data. Ensuring the security and privacy of this data is paramount.

AI in the context of human needs

1. Basic needs

Water, food, warmth, rest and safety: Some machines can produce or identify healthy food and clean water, but the only topics that HMI can effect is the feeling of security and safety, even if it is just a designed feeling.

In the TV series The Twilight Zone, episode The old man in the cave from 1959, after a nuclear war, a group of people is enabled to survive in an irradiated world by the shared informations of a computer about which food is already irradiated and which isn`t. At the end of the episode, the computer is destroyed by a crowd of people who developed the feeling to be oppressed and ruled by the computer, and die in result from eating irradiated food.

2. Psychological needs

Intimate relationships and friends, prestige and feeling of accomplishment. Well designed emotional HMI and conversations, positive feedback and appreciative communication can help to activate a sense of coherence with the environment and the machine as part of it.

A negative example can be found in the TV series The Twilight Zone, episode From Agnes — With Love, where a “female” computer (AI) named Agnes observes a software engineer in interaction with women and starts giving relationship advices to him. Then the computer pretends to be in love herself with the engineer and starts behaving strongly illogically. In result, one engineer after another have to give up working, after having a nervous breakdown, so each time an even better engineer has to step in, that might manage to handle her.

3. Self-fulfillment needs

One of the current trends is to stop or even reverse aging, also by using AI to analyze the effects of behavior on health via large amounts of data. But there is also atrend called “Digital Afterlife”, in which AI-based avatars of deceased people are offered as conversation partners for the bereaved.

An example for “Digital Afterlife” can be found in the series The Twilight Zone, episode Uncle Simon. A spiteful uncle and his niece, who cares for him, do not get along well with each other. Not knowing that the uncle is working on a solution to continue to have an impact on the world after his death, the niece refuses to help her uncle who is about to die after a fall on the stairs, so that she can inherit more quickly. Before his death, however, the uncle was able transferred his knowledge and will into a robot, which treats the niece in the same horrible way as the uncle did before.

4. Social impact needs

The ability to have a positive impact to the coexistence determining system and environment of many people, could be a core skill in times of overlapping crises and could be enabled by machines. In the end, the creation of AI helps some people also to understand themselves better.

While AI is a megatrend, it carries the risk that many jobs could be replaced by it, an example for that can be found in the series The Twilight Zone, episode The Brain Center At Whipple’s. At the beginning of the episode, a manager at Whipple’s calculates how much money and time can be saved by a machine (AI) that he has constructed, which costs only 4 cents per hour for electricity. By eliminating the employees, there would be no more coffee breaks, no sick leave, no maternity leave, etc. But after the last employee voluntarily quits, because he feels lonely at work, the machine breaks down because there is no one left to maintain it.

Archetypes of communication

According to Charles Duhigg, the author of Supercommunicators, there are only three archetypes of conversation:

• Emotional — How do we feel?
• Practical — What is it really about?
• Social — Who are we?

If you don’t know what type of conversation you are having, it is difficult to evaluate the outcome of a communication.

1. Emotional communication

Even it’s clear that R2D2 actually is a robot and he is communicating only with sounds and what he is saying can not be understood, the sounds he is using for communication are composed in patterns that make humans think, he does transmit emotional messages, and the extracted emotions are easily understandable. His sound patterns are e.g. „I have to tell you something urgently“, „I’m disappointed.“ , „I’m excited“. Most viewers do like this character from the first minutes of his appearance.

2. Practical communication

Talking to someone, who does not show his or her true emotions, does make one feel unsure. Darth Vader, who is wearing a mask all the time in the Star Wars movies is a character of this type and has this effect on the audience. Who is he? What does he want? All the time, it is unpredictable what he is doing next. I remember, having seen Star Wars for the first time as a kid, the moment Darth Vader appeared, I thought he was a robot.

3. Social or unsocial communication

Maybe Jar Jar Binks was created as a Stan Laurel kind of character, just without the balancing team partner Oliver Hardy. A reason why the audience learned to hate this character, might be his pure self-focused communication, that results from not listening to others, talking all the time about himself in the third person and in an emotional melody pattern, that makes people feel lucky, that this character is not their roommate.

Accessibility under emotional conditions

The discipline of accessibility already takes into account conditions such as physical disabilities or autism. It would be worth discussing whether there aren’t also emotional hurdles when accessing, for example, machines or situations that can create irrecoverable changes. Can a surgeon who has just negotiated his divorce in the courtroom perform surgery? Should access to support services for people in extreme situations with high emotional stress be simplified and accelerated?

HMI enabled friendliness

According to the futurologist Oona Horx Strathern, modern western society is heading towards an Economy of friendliness. In the professional context, as well as in the private context, the value of friendliness, appreciation, a healthy culture of error, etc. equalizes the monetary value.

Hardly anyone will want to work for a boss or an organization in which there is a lack of respectful interaction with each other. The interaction with friendly and intelligent machines can help us develop these skills.

1. Intense emotions

Within the inner circle, we can find emotions that are more intense. Thinking of AI that is not only tracking phrases and senses, but also emotional states, a patients mood in a hospital for example could be nudged to a more calm mood via a friendly dialog, curated music or other content.

The impact and manipulation of music is well known, it is categorized in the currently most popular usage cases department store music, political music and goosebumps music.

The composer Eydís Evensen for example says „Music is unfair because it determines how we feel.“.

2. Calmed emotions

It is well known that when people are calm, they make better decisions and are better able to deal with difficult situations. So there is an individual, a social and an economic interest in being in a relaxed state as often as possible.

3. Not in contact with ones emotions

Unfortunately, there is another state of mind, in which people are no longer in touch with their emotions or do not allow themselves the full range of emotions. Feelings that do not happen or are not allowed to happen are difficult to measure.

The mental coach Hanna Tempelhagen for example says, that listening to music while jogging distorts the perception of one’s own body signals and can perspectively lead to injuries.