ABOUT THE PROJECT
Periodically I conduct usability testing sessions with members of the XWiki community in order to evaluate the changes we made inside the product. XWiki is an extensible open source platform for developing collaborative applications on top of the wiki paradigm.
I create scenarios focused on gathering data about how different personas use and understand XWiki concepts like: wiki, space, page, application and also how they use the product without prior knowledge of it. The participants can be remote or on location.
Report status shows the success/fail rate of tasks for the participants. This allows rapid identification of problematic tasks. Issues can occur if users are unable to complete the task or if the duration to complete the task exceeds the initial estimation.
Having the user's approval, videos can be recorded in order to further analyse the behaviour together with the development team. Research notes are a great resource, but seeing video can have a greater impact on planning and solving the issues found. Here are some video sample sections from the Scenario #4 interviews:
The most important part of the usability test is identifying and reporting issues, but also providing proposals and solutions for the issues found. If a problem occurs with multiple participants, it needs to be formulated into an issue. Depending on the severity of the issue, solutions are brainstormed or at least held in the backlog.