How do you measure UX (User Experience)?

How do you measure UX (User Experience)?

This is a more thinking/theory post. For a more doing/practical post see “3 New Ways to Measure and Evaluate UX

A good question from Austin Govella, he asked me to blog on it. He and Livia Labate has been pursuing this question at UX Healthcheck. I myself have many thoughts on this question and this post is one thought about how I approach measuring UX for both online and offline experiences. This specifically relates to the UX Value of “Measure the Impact” in the UX Value Mandala.

For a given setting there is a reality, people have experience as they live and work in a particular reality. These people then create or make expressions of their experiences they have in these realities.

Why is it so complicated, can’t we just measure experience?
I am not interested in a reality if no one is experiencing it.
Different people can be in the same reality and yet have different experiences, for example they bring to a reality different expectations and different past experiences. A person’s “experience” is a contstruct and reflection on what happened to them.

Because we are seeking to understand what a certain set of activities or events MEAN to someone we can not just use objective measures of the experience that we collect as the experience unfolds. Too many UX measures soley focus on this one element of measuring UX.

In order to measure UX well we need to collect data on the setting of the experience and on people’s reflection and construction of their experiences.

How to analyze an experience setting.
The setting of an experience should focus on describing elements such as:
1) The actors, 2) their goals, and 3) feelings,
4) the different places, and 5) spaces, and 6) the objects located there,
7) during different events, 8) activities, and 9) acts that happen.

It’s more than just a task analysis.
I hope this expands most people’s conceptualization of user experience as only being made of activities or tasks.
Almost all the UX measurements out there seem to focus at the act and activity level. The better ones even correctly
coallese these into events that people experience, but few that I am aware of look at the other aspects of a experience setting.

Then the expressions that people have of the experience are varied and complicated.
Three good questions to explore are the following:
1 “Why or why not would you recommend this product or service to a friend?”
2 “If your experience with this product or service was a book, what would the chapters be titled?”
3 “What do you think you are going to start to do differently or continue to do the same now that you have had this experience?”

In other words, there are experience potential settings, there are the experiences that people have, and there are the memories/reflections of those experiences which they tell other people about and blog about. A good UX measurement index looks at all three.

Tags: , , ,

3 Responses to “How do you measure UX (User Experience)?”

  1. Michael Carvin says:

    Excellent post, Dennis.

    Your 9 points around the experience setting remind of Stephen Anderson’s definition of Experience Design as “The people, their activities, and the context of those activites.”

    Of course, your point about UX measurement happening most commonly at the act/activity level is valid given that most managers/owners manage what they can measure. As you know, the easiest things to measure are tasks and activities. The real work is in what you propose – measuring the broader whole of the experience. To that end, my sense is that many of the “managers” responsible for the UX fall under either Marketing or IT, and these Experience metrics are nowhere near as important as click-throughs, load times, whatever. (You know my feelings on this subject :))

  2. admin says:

    Michael,
    Thank you for reading and commenting. I like Anderson’s expansion of Experience Design beyond just interactions.

    In Business Anthropology we would say “If you measure it, you will manage it.” I guess I am just advocating that for many “users” an experience is more than just the sequence of acts/activities they do. Especially if your business is concerned with aspects such as brand loyalty and customer retention. Click-throughs, load times, time on site, and those “traces” of observable user behavior are very important, but they if are not already they soon will be just table stakes in the bigger poker game of sustainable competitive advantage.

Leave a Reply