UX Testing Discussion Guide
Introduction and Warm-up
Hi [participant’s name], thank you so much for taking the time to join me today!
My name is [moderator name], and I'm helping with a research project to understand how a new tool called Compass might be able to assist job seekers in their job search journey.
Compass uses research and AI to help job seekers explore their skills and provides guidance on how best to leverage them in their job search.
Your feedback will help us make improvements to Compass so it can work better for job seekers like yourself.
Participant's consent: did you get a chance to sign the consent form?
Everything you share in this session will be kept confidential. We won't use your name or any identifying details in our reports.
So please feel comfortable sharing openly and honestly
Also, there are no right or wrong answers, the most helpful to us is to hear your perspective
Please also keep everything you see and everything we talk about today confidential
Permission to record: I have a small request; would it be okay with you if I record our session?
I’ll be engaged in the conversation with you and want to make sure I don’t miss anything important that you tell me, only myself and the team will be able to see it and we delete everything after the completion of the study
Permission to livestream: I have one more request; would it be okay with you if I livestream our session?
Members of the team working on Compass are eager to learn from your experience and they would love to observe the session.
Thank you so much, much appreciated [if they say yes]
No worries at all [if they say no]
Do you have any questions for me?
Before we jump into trying out Compass, I'd love to hear a bit about your experience looking for a job.
Can you briefly tell me a little about your job search so far?
Thank you for sharing a bit about your job search journey, let’s get started with Compass.
Tasks
Task 1: Initial Interaction [10 min]
Scenario [Moderator to read this part]:
You are a job seeker exploring finding a job. You want to find work that suits you, based on your skillset. Start a conversation with Compass to identify your skills.
Instructions: [Moderator to read this part] (to avoid bias, alternate between the two options as you go through the interviews)
Option 1: how would you go about using Compass to identify your skills.
Option 2: introduce yourself and tell Compass you are looking to identify your skills for your job search.
Feedback: [Moderator to read this part]:
What do you think of the conversation flow so far?
On a scale of 1 to 5, how satisfied are you with this conversation with Compass so far? (1 unhelpful 5 very helpful)
Success criteria: participant successfully initiates a conversation with Compass.
Task 2: Skill Identification [10 min]
Scenario [Moderator to read this part]:
As you can see, Compass has asked you questions about your various experiences
Instructions: [Moderator to read this part]:
Go ahead and answer Compass's questions about your past roles and responsibilities to the best of your ability
Feedback: [Moderator to read this part]:
What do you think of how Compass is trying to identify your skills?
On a scale of 1 to 5, how satisfied are you with this conversation with Compass so far? (1 unhelpful 5 very helpful)
Success criteria: participant answers questions about their work experience and engages in a dialogue with Compass
Task 3: Skill Review & Feedback [10 min]
Scenario [Moderator to read this part]:
As you can see, Compass has generated a list of your potential skills
Instructions: [Moderator to read this part]:
Go ahead and take a moment to review these skills identified by Compass
Feedback: [Moderator to read this part]:
How accurate/relevant do you feel the skills Compass identified for you are? Why or why not?
On a scale of 1 to 5, how helpful was Compass in identifying your skills? (1 unhelpful, 5 very helpful)
On a scale of 1 to 5, how satisfied would you say you are with Compass’ ability to identify your skills?
Success criteria: participant reviews the generated skills, provides feedback, and clarifies any discrepancies
Post-Test Feedback Questions [10 min]
[Ease of use] How easy was it to interact with Compass and understand its responses?
[Accuracy & relevance] How accurate and relevant do you feel the identified skills are? Why?
[Accuracy & relevance] Are there any skills that Compass incorrectly identified for you?
[Accuracy & relevance] Are there any skills you have that Compass did not identify?
Did Compass help you gain a clearer understanding of your skills? why or why not?
Would you recommend Compass to other job seekers? Why or why not?
In the future, Compass will provide you with advice on which sectors you could focus on for your job searches based on the skills it will have identified for you. What do you think about this?
In the future, Compass will match you with jobs that align with your identified skills. What do you think about this?
Any other feedback or suggestions for improving Compass?
Thank you and close
Last updated