OVERVIEW
Microsoft created the Microsoft Learning website (http://www.microsoft.com/learning) to educate and certify general users, developers, and IT professionals on the use of Microsoft products. The site revolves around a set of standardized tests that visitors can take to validate their understanding of a specific Microsoft application or service. Passing these exams, earns the visitor a Microsoft Certification in the topic, which, according to Microsoft Learning, is a standard that is recognized worldwide.
The Microsoft Learning team was interested in the issues first time visitors of their website run into while searching for training and certification exams. We worked with Microsoft Learning on an evaluative study to understand if first time visitors are able to find the training and certification exams that are appropriate for them.
Our usability study consisted of two scenarios that covered four tasks which are frequently completed on the site. We ran seven sessions, including a pilot.
My Role
i was involved in all the stages of the processs. However, I was majorly involved in the following:
- Creating the personas for narrowing down our audience
- Creating the task list for the usability study
- Moderating the usability study and taking notes
- Ananlysis of data
- Creating the task list for the usability study
- Moderating the usability study and taking notes
- Ananlysis of data
Usability Study Goals
We worked toward pinpointing areas where we can improve the search and discovery of trainings & certifications. We concentrated on improving information discovery through a better navigation experience, concentrating on the first time user. At the end of the study we also suggested enhancements to help improve the product’s usability.
The Process
In order to achieve the above goals, we pursued the following process:
(1) We worked with Microsoft to gather actual data about users.
(2) Based on this data, we created user groups. These user groups helped us in creating our personas for the system.
(3) We then crafted a screening questionnaire and began screening participants for our study.
(4) Next, we created our task list and conducted studies using the “think aloud protocol”.
(5) We analysed the data from our usability study and carefully determined how, when & why users fail to discover trainings & certifications.
(6) Lastly, we presented our recommendations to enhance the first time user’s navigation experience.
(2) Based on this data, we created user groups. These user groups helped us in creating our personas for the system.
(3) We then crafted a screening questionnaire and began screening participants for our study.
(4) Next, we created our task list and conducted studies using the “think aloud protocol”.
(5) We analysed the data from our usability study and carefully determined how, when & why users fail to discover trainings & certifications.
(6) Lastly, we presented our recommendations to enhance the first time user’s navigation experience.
ERSONAS:
SCHEDULE:
We carefully selected the personas for our study based on the demographics of users who have visited the site in the past. This helped us in creating a baseline about what certain personas will do when confronted with different tasks on the website and to understand how we can make the experience better.
SCREENER DEVELOPMENT:
We developed our screener to map our sample against it and select participants by considering factors such as eliminating professional user research participants, minimizing no-shows, eliminating the strong -silent types, learning about their familiarity with Microsoft Learning website and ensuring that people are physically able to participate. We also asked questions like “how many hours they spend on the Internet per week” to ensure that participants’ experience with the computer and the Web matches that of our user groups as we didn't want participants to confuse user interface problems with the problems new computer users face in general.
We also aimed to be precise and drafted the sequence of our screener to get to the point quickly, asking the elimination questions first and avoiding questions that are strictly for information gathering. We also asked “what they do on the Internet” to learn about their Internet knowledge as our preference was participants who are more tech savvy.
RECRUITMENT:
We received screeners from 25 participants. We reached out to those who matched our criteria of target demographics and user profiles best and would be available on the days of the study. In the final group of participants who qualified, there were six men and one woman (this includes one pilot participant). As we proceeded with the study, we found that this number was adequate. By the time we reached the fourth participant, most of the major issues were uncovered and being repeated in subsequent sessions.
THINK-ALOUD SESSIONS:
We picked the think aloud method for testing the Microsoft Learning website. Our project required us to test how easy it is for a first time user to navigate the website and find information on it. Thinking aloud suits this kind of project the best because participants will be speaking their thoughts. This allowed us to understand what the participant is actually thinking instead of us guessing.
SCHEDULE:
Each 60-minute session was broken into four sections:
· Pre-session interview
· A warm up task
· Two sets of scenario-based tasks completed with the website
· Post-session interview and debriefAFFINITY DIAGRAMMING:
We used affinity diagramming to find clusters of problem areas and sort them into related groups and sub-groups. This also helped us in finding areas with most severe problems.
RESULTS:
The study revealed some interesting findings and we were able to complement our quantitative data with verbatim comments from our participants. From these comments, we were able to dive deeper into the issues that caused task failure or difficulty.
