Examples of Usability Testing
Example Case: Wearable Fitness Tracker
Example Case: Wearable Fitness Tracker
Introduction:
This is an outline of a baseline user/usability assessment, how I would approach it, and which questions I would ask. I recently used this as a teaching tool for students. Students used fitness trackers, chose data produced by the fitness tracker and developed user tests to assess the device's accuracy.
User surveys:
Will be assessed for brevity, clarity, and producing valuable information. I will provide action steps to improving valuable feedback from users. In cases where surveys reference websites or apps, I will provide annotated screenshots and/or videocasts of clarifying action steps.
Will be assessed for brevity, clarity, and producing valuable information. I will provide action steps to improving valuable feedback from users. In cases where surveys reference websites or apps, I will provide annotated screenshots and/or videocasts of clarifying action steps.
Examples of poor surveys that do not result in quality feedback include:
How would you rate your overall experience of this product?
Would you recommend this product to a friend or family member?
How would you rate your overall experience of this product?
Would you recommend this product to a friend or family member?
Examples of quality surveys producing useful feedback include:
When you put the wrist tracker on your arm, how easy or difficult was this process?
When you attempted to connect the device to the app, did this happen automatically, or as expected, or do you feel extra unnecessary steps were required?
When you put the wrist tracker on your arm, how easy or difficult was this process?
When you attempted to connect the device to the app, did this happen automatically, or as expected, or do you feel extra unnecessary steps were required?
The fitness tracker’s accelerometer tracks movement data and presents it to you as “steps”. How do “steps” register with you? Is it clear what the purpose of this data is?
Do you feel that increasing the daily steps improves your overall quality of life and how direct is this connection?
Do you feel that increasing the daily steps improves your overall quality of life and how direct is this connection?
Usability tests:
Will be assessed for goal assessment, and producing valuable feedback from users. I will provide action steps for improving the test, looking for improved user feedback during the tests. I will provide any screenshots or videocasts necessary to clarify these action steps.
Will be assessed for goal assessment, and producing valuable feedback from users. I will provide action steps for improving the test, looking for improved user feedback during the tests. I will provide any screenshots or videocasts necessary to clarify these action steps.
Examples of poor usability tests include:
Please take this fitness tracker home and come back and let us know what you think.
Your goal is to connect the fitness tracker to your cellular app. We will document your on screen processes.
Please take this fitness tracker home and come back and let us know what you think.
Your goal is to connect the fitness tracker to your cellular app. We will document your on screen processes.
Examples of quality surveys producing useful feedback:
Your goal is to connect the fitness tracker to your cellular app. We will document your on screen processes, record video of you during this process, and ask you to talk us through your thinking during the process. Afterward we will ask questions on your decision making.
Your goal is to connect the fitness tracker to your cellular app. We will document your on screen processes, record video of you during this process, and ask you to talk us through your thinking during the process. Afterward we will ask questions on your decision making.
Usability study:
Will be assessed first for human subject factors, is the test a violation?
Assessed for empirical approaches, measuring quality of data, and reproducibility of the test data. Other issues that will be assessed, the inclusion of money in the user test (sometimes appropriate, sometimes not), any bias in the study will be assessed (is it biased toward improving the sale of a specific product?). I will provide meaningful feedback, a second eye to note any missing perspectives or possibilities.
Will be assessed first for human subject factors, is the test a violation?
Assessed for empirical approaches, measuring quality of data, and reproducibility of the test data. Other issues that will be assessed, the inclusion of money in the user test (sometimes appropriate, sometimes not), any bias in the study will be assessed (is it biased toward improving the sale of a specific product?). I will provide meaningful feedback, a second eye to note any missing perspectives or possibilities.
Example:
A recent ACM article suggests that wearable devices are experiencing a market loss because users do not seem to hold long term affect with the data produced by them.
A usability study might assume that a user understands fitness tracker steps emotionally, and an emotional attachment such as “success”, “achievement”, “pride”, or “accomplishment” is felt when that step count is achieved.
A recent ACM article suggests that wearable devices are experiencing a market loss because users do not seem to hold long term affect with the data produced by them.
A usability study might assume that a user understands fitness tracker steps emotionally, and an emotional attachment such as “success”, “achievement”, “pride”, or “accomplishment” is felt when that step count is achieved.
Many usability studies focus on user and device, without considering overarching conditions between direct user - device interactions. An improvement to this study might include understanding which emotion/word associations users tend toward, and understanding when the data produced is not only accurate, but also elicits this emotional response. The goal is to understand a user’s emotional response to the data and therefore the experience of using the device.
A poor example of developing a test population is to not account for the “no difference” assumption. Users should be tested prior to taking this test to ensure they are all capable of achieving the same results. Can they all put the fitness tracker on, connect it to a cell phone app, and understand the data? Is time a factor, and will it require some populations longer to complete these tasks? Was the sample population chosen based on "who was available" (typical of many marketing surveys), or based on the above criteria?
© Amanda Ervin 2019
