Figure 1: Evidence Maps Homepage
I planned, conducted, and reported usability testing for a Search Tool, used by medical professionals. The tool helps users search scientific evidence about medical conditions and treatment in articles reviewed by an internal team.
American Speech-Language-Hearing Association (ASHA). Washington, DC, USA.
Challenges related to creating meaningful usability tasks for medical professionals (without being one) were solved by teamwork with content experts stakeholders.
I collaborated with stakeholders to define project objectives. The Evidence Maps tool went through significant design changes the year before the project start, and the team wanted to understand what works and what does not work with the new UX. We identified the following objectives:
• Identify the main usability issues in using the search tool for scientific insights
• Recommend UX changes
I set up the Crazyegg Heat-map tool by using Google Tag Manager to collect the (quantitative) usage data. This quantitative data is required to support the qualitative data gathered from usability testing.
Figure 2: Heat-map Tool Results Example (Used in Multiple Projects)
Performing usability testing comprised the following tasks:
The tool targeted Speech-Language Pathologists (SLPs) and Audiologists, which are medical professionals with lengthy training background. I was not an SLP or an Audiologist, and so I did not know what may be a typical task in the tool.
I collaborated with the subject matter experts to define the tasks and then created open and closed tasks, and session scripts.
I created one task to be used at the beginning of the study to simulate and understand how users use the Evidence Maps.
Scenario #1
"Reflect back on your own practice. Have you had a patient or student come to you with a disorder that you hadn’t seen recently? Or did a client ask questions about a treatment you haven’t used? Think of your clinical question and tell us about it. Use the Evidence Maps to see if you can find any information to answer your question."
I created five closed tasks for SLPs. I also created five closed tasks for Audiologists.
To view a complete list of tasks created for SLPs, see Tasks for SLPs.
I recruited participants directly from the ASHA website and the ASHA Community forums. I used a screener, created using Ethnio, to screen participants. Each potential participant went through a short introductory screening to collect information about the number of years of experience, work settings, experience with the search tool, and profession. From the pool of potential participants, I selected participants with different levels of expertise in using Evidence Maps and having different work settings. Each participant received $30 for the session.
Figure 3: Participants with Varied Experience and Work Settings
I conducted and moderated online test sessions with participants and project team members. The project team members were invited to join the sessions to see the participants’ responses.
I analyzed the test results and created reports, summarizing the test results by tasks and issues, in the following formats:
Figure 4: Success Rate of the First Open Task
Figure 5: Success Rate of Users in Closed Tasks 2 to 6
Within 2 to 3 tasks, the success rate of the participants went up. Most participants learned how to use the tool within 30 minutes of working with it.
I analyzed the participants’ responses and refined conclusions of top perceived benefits.
"It is a great resource for evidence-based practice – a lot of information!"
"Love the fact it has guidelines that I can use immediately in practice."
"Seeing the conclusions of the article helps to decide whether to read the entire article or not."
"The Search All Maps is great! It helps me in finding information quickly."
Figure 6: Perceived Benefit: Filters Facilitate Quick Results
Two search boxes on the same page created confusion. After trying the first search box and not finding what they were looking for, participants said they would leave the page. I recommended unifying the search boxes to avoid confusion for users.
Figure 7: Issue: Two Search Boxes for Evidence Maps.
Figure 8: Recommendation: Unify the Search Boxes
Some of the terms in the Results Filter area were not very clear. In addition, users could not easily refine the results by using sub-categories.
I recommended using clear terminology for filtering the results and improving the user experience by bringing the sub-categories closer to the parent category. Thus, allowing users to easily select appropriate sub-categories while filtering the results.
Figure 9: Results Filter Area Usability Issue and Recommendation.
Some of the users took a bit longer to use the Search tool. I recommended including Quick Training Tips to shorten the time of first-time users in using the Search tool.
Figure 10: Training Wizard Recommendation
Drop me a line and I will get back to you ASAP.
Or contact me:
Phone: +972-50-9623607
e-mail: ifat.prouser@gmail.com
Oops! Something went wrong while submitting the form :(
