Creating a Usability Test Plan
In this project, you will:
Create a usability test plan
Develop the usability testing script
Run usability test and report on results
In this part of the course project, you will create a plan for your usability test for evaluating the high-fidelity prototype you created in the previous course.
Define the following components of your plan for the usability testing. Use Step One of the Usability Testing Planning Guide to aid your creation of the plan.
You have considered the importance of evaluation goals, usability metrics, participants, time/location, data collection, and materials.
Set Evaluation Goals
Evaluate Flow and Navigation: Check if the participant can easily go from one page to another. Make sure the buttons for 'Next' and 'Back' and the bar at the top showing progress all work well together.
Test Accessibility: Look into whether the prototype is easy to use for everyone, especially those with different needs.
Usability of Form: See how easy it is for the participant to complete the form and get to the results. Make sure the questions are easy to understand.
Navigation Clarity: Check if people can easily see where they are in the process and go back and forth without getting confused.
User Satisfaction: Find out if the participants are happy with the prototype. Ask about what they think of the design, if it's easy to use, and if they have any ideas to make it better.
Define Metrics
Evaluate Flow and Navigation:
- Effectiveness: How well participants can navigate through the pages.
- Efficiency: How quickly participants can complete tasks.
Test Accessibility:
- Effectiveness: How well the prototype works for everyone.
- Satisfaction: Gather user feedback on accessibility
Usability of Form:
- Effectiveness: How well participants can navigate and choose the options.
- Efficiency: How fast participants can complete the form.
Navigation Clarity:
- Learnability: How easily participants understand and move around.
- Error Rate: Track to identify where participants have difficulties in navigation, indicating potential issues with clarity.
User Satisfaction:
- Satisfaction: What participants feel overall about the prototype.
- Memorability: Collect feedback on how participants remember the prototype
The Participant
The person being interviewed is the CEO of a beauty services company with a team of six people. She wants to improve employee engagement and is curious to learn more about what we can offer.
Recruitment Details
Compose the content of the recruitment message. Paste the content of the message below.
Hello, we're launching tests for our employee engagement app and are seeking feedback from managers and CEOs. Your insights are crucial for our research and the success of our product. If interested, please let us know when we can schedule a brief interview. Your participation is highly valued!
Time/Location: The section it will be remote on Tuesday at 9am.
Materials: We'll need a desktop computer to ensure a smooth experience during the usability test. Keep a notepad and pencils handy for any notes you'd like to take. And we will record the session for evaluation purposes.
What would be the most effective recruitment method? In other words, how would you advertise the usability test?
The best way to find people is to start with friends or social media. We can also use platforms like LinkedIn or Twitter, and post on job boards or forums related to the app's topic. This way, we reach a diverse group interested in what the app offers.
Data Collection
What data will be recorded during the usability test session, and how will the data be captured? The data collection technique needs to allow achieving the evaluation goals and apply the metrics defined earlier.
During the usability test, we'll gather important information to understand how well the prototype works. We'll track metrics like time taken for tasks, any navigation errors, and how users feel about accessibility. We'll also focus on the form's usability, checking completion rates and the time users take to fill it. Ensuring clear navigation is crucial, so we'll gather feedback on understanding the progress bar and any issues users face. Lastly, we'll assess user satisfaction using scores and open-ended feedback. This data helps us improve the prototype based on users' experiences.
Set Evaluation Goals
Define Tasks:
Define a set of three to five tasks that the participant will perform during the session, along with task instructions. Use the Evaluation Tasks tool for more specific details on choosing and constructing tasks.
In this part of the course project, you will develop the script for your usability test.
Task One
Task instructions for participant:
Watch the Introduction Video and start the campaign journey.
Task instructions for evaluator
Scenario: The participant will be on the welcome screen, where an explanatory video will be shown with instructions about the product and the prototype. After viewing the video, the participant is expected to click on the "Start" button to commence their journey.
Evaluate Start Button Interaction: As the participant clicks on the "Start" button, observe the speed and confidence of the action. Take note of any hesitations or difficulties encountered in initiating the campaign.
Explain why this task
In this task, we want the participant to engage with the introductory video on the welcome screen. After watching the video, the participant is prompted to click the "Start" button to commence their journey. The goal is to assess how smoothly and confidently the participant initiates the campaign. By evaluating the speed and confidence of the "Start" button interaction, we can try to identify any hesitations or difficulties that may arise during this crucial initiation phase. Understanding the participant's experience at this stage provides valuable insights into the user's initial engagement with the campaign journey.
Task Two
Task instructions for participant:
Answering Campaign Questions: Your goal is to express your preferences for the campaign. Please navigate through the form pages to get the results.
Task instructions for evaluator
Observe the participant's navigation through the form pages. Check the clarity of questions, ease of selecting options, and functionality of the "Next" button. Pay attention to how the participant interacts with the form and note any challenges or confusion in the process.
Explain why this task
By navigating through the form pages, we can evaluate the clarity of the questions, how easily participants select options, and how well the "Next" button functions. This gives us insights into the user experience, helping identify any challenges or confusion in interacting with the form.
Task Three
Task instructions for participant:
Returning and Changing: Please start to answer the questions and, at some point, navigate back to the previous question and change your answer. Observe how the system responds to your actions.
Task instructions for evaluator
Observation: The participant is instructed to start answering the questions and then intentionally go back to a previous question to modify their response. The purpose of this task is to evaluate the system's responsiveness to changes in user input and assess the ease of navigating back and forth in the prototype.
Explain why this task
This task was included to assess the system's ability to handle user interactions dynamically. By asking the participant to change a previous answer, we can evaluate how seamlessly the prototype updates in response to user input. Additionally, this task provides insights into the participant's experience in navigating through the prototype and making adjustments, contributing to a comprehensive usability evaluation.
Task Four
Task instructions for participant:
Canceling and Restart: Please start to answer the questions, cancel them on the way, and initiate the form again.
Task instructions for evaluator
Observe the participant's reaction. Once the participant restarts the previous answers, they will be saved. Check how the participant feels about it. Note any confusion or frustration during the cancellation and restart process.
Explain why this task
This task is about the participant’s reactions and experiences when they decide to cancel their responses midway through the form and then restart. Additionally, noting any confusion or frustration will help to improve the user experience, ensuring a seamless and satisfactory process for users who choose to restart after canceling. This task allows us to refine the prototype based on participant feedback and reactions.
List and compose the questions you will ask before and after the tasks.
Questions Before completing tasks
1. Can you tell us a bit about yourself? Where you live, your interests, and what you find important in life.
2. Have you ever participated in a Usability Test before? If yes, what was it like for you?
3. How large is your team, and do they work remotely or in-person?
4. How excited or curious are you about trying out the prototype, and is there anything specific you hope to discover or explore?
Questions After completing tasks
Task instructions for evaluator
Now that you've experienced the prototype, can you share your thoughts on the overall experience? What aspects stood out to you, and is there anything you found particularly interesting or challenging?
Write a detailed script for your session protocol. Using different font colors, distinguish between parts you will read out to the participant and parts that are instructions to the evaluation team.
Welcome and introduction
Hi, I'm Adriana, how are you? Thanks for joining me in this Usability test. I’m a UX student from eCornell and I’m excited in sharing this prototype to get an understanding how you feel using it, what can we improve in design and functionality, and where it may not meet your needs in its usage. There's no right or wrong here. It’s about how, what and what else should have for a pleasure experience.
This will be a short section, about 20 minutes.
Initial questions
Tell us a bit about yourself—where you live, your interests, and what you find important in life.
Our app focuses on employee engagement, The goal is to improve interaction and collaboration through fun activities. Usability. Have you ever taken part in a usability test before? If yes, what was your experience like? How comfortable are you with technology?
Awesome! Now, I'll introduce you to the app. Thanks a lot for your participation.
This will be a short section, about 20 minutes.
Informed consent script or form
Before we begin, I'd like to go over a few details. Your participation is voluntary, and you can stop at any time. We'll be recording this session for analysis purposes only, and your data will be kept confidential. If you agree to this session, please let us verbally agree.
Prototype interaction instructions
Welcome to Collab Hub's prototype! Start with the Welcome Video on the first screen. Trust your instincts as you interact with the prototype. Just a heads up, not all features are fully ready—it's a quick check on your interaction and language feedback. Enjoy the journey!
Debriefing and thank you
Thank you for your time in participating. Your input is what makes the difference.
Run your usability test based on the plan and script you created in Part One and Two. Recruit one participant for the test. Record the session to make sure you’ve captured everything and have it available for analysis. Use Step Three of the Usability Testing Planning Guide tool as you run the test and collect data.
Analyze the data, following the guidelines in the Usability Testing Data Analysis tool. Develop a list of design problems, suggest solutions to these problems, and decide on the design problems you will fix. Submit your report along with any raw data from the test.
Run a Usability Test
Recruit one participant who fits the selection criteria of the user group you designed for. Follow your usability test protocol and script. Record your session. Submit a file, or multiple files, that includes the raw data from the session: the notes you’ve taken during the session, the video or audio recording, and/or the video screen capture.
Video is not available for the public.
Analyze the data and apply metrics
Evaluate Flow and Navigation:
- Effectiveness: The participant completed tasks quickly, indicating a smooth flow through the prototype.
- Efficiency: The endpoint of the first task was not clear, causing a delay in starting the second task. Improved clarity between tasks is needed.
Usability of Form:
- Effectiveness: The participant navigated and chose options well, contributing to a positive experience.
- Efficiency: The form's usability met expectations, with the participant appreciating the casual and open-answer format.
Usability of Form:
- Effectiveness: The participant navigated and chose options well, contributing to a positive experience.
- Efficiency: The form's usability met expectations, with the participant appreciating the casual and open-answer format.
Navigation Clarity:
- Learnability: The participant understood the status bar, indicating success in this aspect.
- Error Rate: The participant didn't see the error on top for a functionality that is not working.
User Satisfaction:
- Satisfaction: Overall, the participant felt good and clear about the tasks and the prototype.
- Memorability: The participant's positive sentiment suggests a memorable and user-friendly experience.
In summary, the data provided valuable insights into the effectiveness and efficiency of the prototype. While the participant encountered challenges with task clarity and instructions, the overall positive experience and satisfaction with the form design indicate a strong foundation. Improvements in task transitions and clarity can further enhance the prototype's usability.
UX Problem Instances
UPI1
Observation or data point:
- Welcome Video
Why is this a UX Problem instance?
- Some users just ignored the Video and moved on without learning about the product.
UPI1
Observation or data point:
- Welcome Video
Why is this a UX Problem instance?
- Some users just ignored the Video and moved on without learning about the product.
UPI2
Observation or data point:
- On the first form page
Why is this a UX Problem instance?
- Some users point out that the status bar is clickable.
UPI3
Observation or data point:
- The image on the 3rd question
Why is this a UX Problem instance?
- It doesn't convey the intended emotions; it feels a bit unclear and lacks engagement.
UPI1
Observation or data point:
- Welcome Video
Why is this a UX Problem instance?
- Some users just ignored the Video and moved on without learning about the product.
UPI4
Observation or data point:
- The results page
Why is this a UX Problem instance?
- The participant felt that he didn't have enough information to continue.
UPI5
Observation or data point:
- Hitting the Back button
Why is this a UX Problem instance?
- Some of the users would like that when you hit back it would clean up the answer.
UPI6
Observation or data point:
- Clean up the previous answers
Why is this a UX Problem instance?
- The app is not cleaning up the answers between the steps.
UX Design Problems
UDP1
Problem description:
- The user didn’t see that he suppose to click on the video to start it
Based on which UPI’s: UPI 1
Severity: Major
Suggested Solution: Just add a message and a PLAY button on the video screen
Solution Cost: Easy
Make Changes(yes/no): Yes
UDP2
Problem description:
- The status navigation bar should not be clickable
Based on which UPI’s: UPI 2
Severity: Medium
Suggested Solution: Disable the navigation bar status bar
Solution Cost: Easy
Make Changes(yes/no): Yes
UDP3
Problem description:
- The options for the first question
Based on which UPI’s:
Severity: Low
Suggested Solution: One participant mentioned that their intention is to implement the prototype within a small team, even though they are part of a larger team. They would like to test it initially in a smaller team setting.
Solution Cost: Easy
Make Changes(yes/no): Maybe
UDP4
Problem description:
- Hitting the Back button
Based on which UPI’s: UPI 5
Severity: Low
Suggested Solution: Some of the users would like that when you hit back it would clean up the answer.
Solution Cost: Easy
Make Changes(yes/no): No
UDP5
Problem description:
- On the next step
Based on which UPI’s: UPI 7
Severity: Low
Suggested Solution: Some users point it out that the previous answers was not reset
Solution Cost: Easy
Make Changes(yes/no): Yes
© 2024 by Learning Matrix. Empowering Minds!