UX Research Methods

Direct contact with participants one-on-one to collect firsthand personal accounts of experience, opinions, attitudes, and perceptions

Interviews are most suitable for exploratory research at the beginning of a research project where the problem space needs to be further defined. Because of the self-report bias that comes with interviews, it is best when this method can be triangulated with other methods that offer more insight into user behaviors. Interviews can be either structured with a set list of questions or unstructured, where the interviewer comes with a list of themes or topics to guide the conversation, but leaves room for deeper probing.

Steps for Conducting Interviews:

1. Define your general research question and problem space.
2. Identify who would be best to interview. Are these stakeholder interviews? User interviews? Are there subsets of users? Aim for 4-8 interviewees within each subset or category that you identify.
3. Develop an interview guide.For semi-structured interviews, begin by outlining topics or themes you want to explore with the participant. Once you’ve narrowed down 3-4 themes, draft 3-6 questions for each theme. Don’t forget to add introduction and conclusion sections to your interview guide where you give an overview of the project, ask for consent, and wrap-up the conversation.
4. Conduct the interviews.Come with your interview guide handy and a system for taking notes. Use a software that will record audio and/or video of the conversation (after receiving participant consent to be recorded). It is best to conduct interviews with a teammate where one person can moderate and the other can capture notes.
5. Debrief the interview. Take 15-20 minutes after the interview to debrief with your teammate top takeaways and key findings from the interview while it’s fresh in your minds.

Attentive observation of people, artifacts, environments, events, behaviors, and interactions

Ethnography involves semi-structured observation in the exploratory phase of the design process. Observing users in their natural environments is ideal to get an honest understanding of users’ behaviors. Generally, ethnographic methods enable researchers to observe with an open mind, but they may come with certain items they are looking for to structure the process to codify certain behaviors.

Steps for Conducting Ethnography:

1. Identify the problem space you’re working in and your research question.
2. Identify where the activities you’re interested in studying naturally take place. (i.e. If you’re interested in people’s grocery shopping behaviors, conduct ethnography at a grocery store.)
3. Find a location and time that will be ideal for your observation.Consider when the place is likely to be more or less populated depending on your research question.
4. Develop an observation guide. Consider what types of activities or areas of the environment you’re hoping to observe. Are there certain behaviors you can codify and look out for?
5. Gather the necessary equipment. Bring your observation guide, a camera to document your observations, and a way to take notes about what you are observing.
6. Go to the location and conduct observations. Spend a few hours observing user behavior in this space and documenting what you see. Pay attention to cues beyond visual ones such as what you’re hearing or smelling.
7. Debrief the observation session After completing an observation session, debrief top takeaways with your team.

An immersive, contextual method of observing and interviewing that reveals underlying (and invisible) work structure

Contextual inquiry is an immersive method, which allows the researcher to probe while also observing the participant in the context in which they typically perform the activity they want to study. Unlike a typical interview where the interviewer is moderating, contextual inquiry adopts a master/apprentice model where the interviewee is the master and the interviewer is the apprentice. This enables the interviewer to truly be immersed in the user’s daily routine, learning from the “master” about how they do things while observing and asking probing questions.

Steps for Conducting Contextual Inquiry:

1. Define your research questions and problem space.
2. Identify and recruit users who fit the profile for users you’re interested in studying within the problem space.
3. Develop a guide for contextual inquiry.Consider when the place is likely to be more or less populated depending on your research question.
4. Develop an observation guide. Plan a clear introduction where you ask for the participant’s consent and give an overview of your project. Note any exploratory questions you are interested in asking about. Include a conclusion to wrap-up the session.
5. Conduct the contextual inquiry. Go to the user’s environment, introduce yourself, and let them carry out their typical activities while you observe and ask questions at unobtrusive times. Record the session with audio and/or visual recording devices. Note observations and capture images of the user’s environment.
6. Interpret your notes. After the contextual inquiries, conduct interpretation sessions with your teammates where you share back your raw observations. Use these interpretation notes for an affinity diagramming exercise to identify key themes.

Guiding artifacts and activities that allow users to provide snapshots into their daily lives and events

Diary studies are beneficial for capturing information from users across a longer period of time, sampling their thoughts, feelings, and behaviors at key moments. Traditionally, diary studies have involved sending participants diaries with activities written in them for the participant to complete at certain points throughout their days. Each page entry in the diary is guided with a question or prompt, and creativity is encouraged for participants to include images, drawing, or text. A reflection space is also typically included.

With the rise of digital tools, many platforms now exist to support conducting diary studies online. These enable the researcher to set up activities for a user and release new prompts each day or week for the duration of the study. Users can record audio and video of certain moments, upload images, and respond to tasks. Some of these platforms include dscout, Recollective, and Lookback.

Steps for Conducting Diary Studies:

1. Define your research question and what you hope to learn.
2. Identify and recruit users who fit the profiles you hope to study. The amount of users you recruit will depend on your research question, time, and budget. However, diary studies can be expensive due to their length and can potentially result in attrition over time. Consider these aspects when deciding how many users to recruit.
3. Develop the diary. Either on a digital platform or with a paper diary, break the time frame of the study down to determine what topics you want to focus on at which point during the study. Also determine when you want participants to complete the diary. Daily? Weekly? Retrospectively at the end of each day? Each time they complete said activity?
4. Create the activities. When you’ve decided when you want participants to complete the diary each day and which topics you want to explore at which parts of the duration of the study, craft activities related to each topic. For each entry, make sure to start with a guide or prompt for the user and balance how time-intensive tasks are for each day. Include any clarifying or probing questions as needed.
5. Deploy your diary study. Either send the paper diaries or the link to the digital diary to participants. Be on hand to answer any questions as participants get onboarded. Set the time frame for when activities will be released for digital studies.
6. Track responses. If using a digital platform, track participant diary entries throughout the study, so you can probe more deeply if you have a question about a response.
7. Synthesize the data. At checkpoints throughout your study and at the end, begin sifting through the data to identify patterns, themes and key takeaways.

Testing the usability, learnability, and desirability of competitors' products as benchmarks

Competitive analysis is an exploratory research method that enables researchers to keep track of competition’s business activity. When exploring a new problem space, research teams can see what similar products or services currently exist. To understand their strengths and weaknesses, researchers can conduct usability tests or other research methods using competitor tools to better gauge where their competition stands.

Steps for Conducting Competitive Analysis:

1. Define your problem space.
2. Explore who else is offering similar products or services. If possible, test out those products and services yourself, noting key features, strengths, and weaknesses. Identify and recruit users who fit the population your project caters to.
3. Conduct competitive testing. Schedule sessions with recruited users where you explore competitor products. Potentially conduct a usability test on competitor products, noting points of delight, confusion, or frustration.
4. Debrief sessions. After each competing testing session, debrief findings and key takeaways with your teammates.
5. Synthesize information. Compile information gathered about competitors to help inform your company’s competitive advantage and differentiator, as well as areas of improvement to match competition.

Asking users to sort components into different buckets based on their comprehension and expected categorization

Card sorting is beneficial when trying to understand users’ comprehension and expected organization of content. While the exact categories and format of card sorting may vary depending on the research question, card sorting involves providing users with many cards containing concepts, terms, or features and asking them to sort them into categories. Card sorts can either be structured, where the categories are predefined by the researcher, or unstructured, where participants can label categories for each grouping of cards as they see fit. This method is often used when determining the navigation of a website or application, the organization of content on a site, or prioritizing hierarchy of features. Many digital tools now enable card sorting such as Optimal Workshop, Recollective, and User Zoom.

Steps for Conducting Card Sorting:

1. Define your problem space and research question.
2. Decide if you will do a digital or in-person card sort. Select a digital platform to use if you opt for a digital card sort.
3. Identify and recruit users that fit the profile you’re studying.
4. Develop cards and categories if conducting a structured card sort. List only one category, concept, term, or feature on each card. Cards can include images, sketches, and text, but keep the text pithy.
5. Conduct the card sorting sessions. Work iteratively with individual participants or small groups of participants (no more than 3-5 people) according to the book Universal Methods of Design. Conduct no more than 15 card sorting sessions.
6. Prompt the users to sort the cards. Provide instructions for users to sort the cards and leave them time to work through all the cards (~30-100). Ask them to think aloud while sorting so you understand their thought processes.
7. Include the option to add additional cards where needed.
8. Synthesize data. Notice patterns in which categories participated sorted different cards into. Online tools will create graphs and visualizations of how participants categorized the cards. Use this information to inform your experience design project.

A human-centered approach incorporating active user and stakeholder engagement in participatory design exercises

Co-design sessions bring users and stakeholders into the design process as collaborators. Using interactive activities such as cultural probes, diary studies, collages, and design workshops, users can create design concepts alongside the design team.

Steps for Conducting Co-Design Sessions:

1. Identify and recruit users who fit the profile you’re studying.
2. Plan participatory design activities such as cultural probes, collages, or design workshops. Prep any materials needed such as large papers, markers, pre-cut collage items. Define instructions for the workshop.
3. Conduct co-design sessions. Invite a small group of participants to the session if conducting a design workshop and provide the creative toolkits. As they work through their designs, probe around why they are designing things certain ways.
4. Collect the designs. Bring back the materials from the co-design sessions and identify similarities and differences across the examples with your team. Use these insights to inform your design process.

Convening multiple users to provide insight into themes, patterns, and trends experienced collectively by groups

Focus groups are less commonly used as a user research method as participants in focus groups can be susceptible to groupthink, where they change their real views to match what the rest of the crowd thinks or says. Focus groups are a successful user research method for studying user experience with activities typically performed or experienced in group settings such as redesigning a workplace environment used by many employees. This method can also be used by market researchers to understand opinions or perceptions of a group potentially about a brand or product.

Steps for Conducting Focus Groups:

1. Define your research question and what you hope to answer through conducting a focus group.
2. Identify and recruit participants that fit in the problem space you are studying.
3. Prepare a guide and materials for the focus groups.Identify 3-4 topics you want to discuss with participants and 4-5 questions per each topic. Consider bringing in interactive activities, such as voting where something fits on a spectrum or on a mood board, postcards from the future exercise where participants envision the future state of an experience, or a homework exercise asking participants to bring an image of an experience.
4. Conduct the focus groups. At the set time, gather the participants in a room. Bring equipment to record audio and video of the session as well as all your materials and name cards for the participants. It’s ideal to conduct the session with one moderator and a teammate who can help pass out materials and take notes.
5. Synthesize data from the focus groups. Watch through recordings and note key themes and takeaways from the sessions.

Comparing multiple design concepts in quick succession to learn how people react while also validating user needs

Taken from its real-life counterpart, speed dating is a method where researchers rapidly test design opportunities with potential users. Users are exposed in rapid succession to potential design ideas in the form of storyboards or simulated environments to validate the need that the proposed concepts solve. This method is typically used after conducting other contextual field methods and beginning to ideate around potential design opportunities.

Steps for Conducting Speed Dating Sessions:

1. Ideate potential design opportunities as a team and vote on the most successful ones.
2. Create storyboards for each scenario. Include humans in the storyboards and don’t focus on technical features of the design. Scenarios should be centered around solving specific user needs.
3. Create a cover page for each storyboard. Cover pages should be used by the researcher and not shown to participants. The cover page serves as a guide to summarize the user's needs and follow-up questions.
4. “Speed date” storyboards in a session. Present each storyboard to an individual or small group in a series fashion followed by focused follow-up questions. Ask the users to rank how accurately the storyboards represent their needs and the effectiveness of the proposed opportunity.
5. Debrief the speed dating sessions. Discuss findings and takeaways from the storyboard sessions. Refocus needs as were validated by the sessions, refine scenarios, and consider additional design opportunities.

A method where participants verbalize what they are doing and thinking while completing a task, revealing interface aspects that delight, confuse, and frustrate

Think aloud protocol is a technique that can be used in a range of user research methods including usability testing, contextual inquiries, and card sorting. The technique involves asking users to think aloud as they complete a task or activity whether that task was prompted by the moderator as in a usability test or is one they are doing organically as in a contextual inquiry.

Steps for Conducting a Think Aloud Study:

1. Identify and recruit users that fit your target audience.
2. Develop a guide of tasks you want them to complete while thinking aloud.
3. Ask the user to think aloud. Begin the session by prefacing that thinking aloud is unfamiliar for most participants, but it will help you gain insight into their thought processes and reactions. Don’t be afraid to prompt users if they stop thinking aloud at any point during the session.
4. Debrief the studies with your team. Identify key themes and takeaways from the study.

Focuses on people and their tasks and seeks empirical evidence about how to improve the usability of an interface

A method which employs the think aloud protocol, usability testing occurs later in the design process after a working prototype has been developed. Usability testing sessions are one-on-one engagements where participants are asking to use an interface to complete a series of tasks and think aloud while doing so. The moderator may ask probing follow-up questions. This method helps design teams identify what might be delightful, confusing, or frustrating about a design before the product is formally launched.

Steps for Conducting Usability Testing:

1. Develop a working prototype. Create a functional prototype of your design that at least captures the scenarios you want to test. To do this, you can use design tools such as Figma, Sketch, or InVision, or use programming languages such as HTML, CSS, and Javascript.
2. Develop a usability testing guide. Select which tasks you want to have users complete. Write out detailed tasks, scenarios, and follow-up questions for each. Also, formulate a system for taking notes and noting usability issues, potentially using a rainbow spreadsheet.
3. Conduct the usability test. Prompt participants with the tasks and time how long it takes for them to complete each. Have them think aloud while completing. Ask additional follow-up questions and have them rate the usability of the product to accomplish each task after they’ve completed it.
4. Synthesize information from the usability tests. Capture recurring usability issues that happened across participants to prioritize which features to fix first. Make these updates in the design and implementation.

Analyzing interfaces with respect to a set of usability best practices to detect usability problems before actual users engage with the interface

An informal usability method, heuristic analysis is an evaluation of interfaces before deployment to identify usability issues as defined by a set of usability standards, Nielsen’s 1994 Heuristics. Unlike usability testing, this method of interface evaluation is conducted by a design team rather than actual users.

Steps for Conducting Heuristic Analysis:

1. Gather team members to conduct heuristic analysis. Around five teammates is an ideal number for capturing the bulk of usability issues. Team members should range from novices to experts in UX and programming.
2. Provide evaluators with materials. Evaluators should have the link to the prototype as well as a reference of Nielsen’s 1994 Heuristics.
3. Conduct heuristic evaluation. Have evaluators complete tasks on the interface while noting instances in violation of the heuristics and marking those down.
4. Prioritize heuristic issues. Triage the severity of violations to address the most important issues first before launching.

Comparing two versions of the same design to see which performs statistically better against a predetermined goal

A/B testing is a quantitative, optimization method enabling researchers to gather statistically significant information about which versions of a design perform better. These are often used for design questions that are not intuitive as to which would perform better for a user. Participants are randomly assigned to an “A” test or “B” test to determine which design performs better.

Steps for Conducting A/B Testing:

1. Define your research question. Which two versions are you hoping to test? Are you testing color? Text or copy? Hierarchy? Plac
2. Develop two versions of your design. Create an “A” test and a “B” test.
3. Randomly assign participants. Participants either receive test “A” or test “B”.
4. Analyze the results. Look at certain metrics such as click-through rates, purchase rates, or completion time, among others, to determine which test performed better.

Measurement, collection, analysis, and reporting of internet data to understand and optimize usage and usability

A form of digital trace data, web analytics are used to capture real quantitative information about how users are interacting with a specific product, website, or service. Using tools such as Google Analytics, FullStory, Heap Analytics, or others, teams can track user engagement and activity on the site. This informs product teams about potential business issues or opportunities and potentially challenges in the usability of the site.

Steps for Working with Web Analytics:

1. Implement analytics tools. A range of analytics tools are available for implementation in your digital site. Install these on the back-end before deploying your product.
2. Define metrics. Identify key business and product metrics you want to track. Business metrics relate to click-through rates and purchases while product metrics related to user experience and activity such as # of active users, # of new users, or time spent on the site. These metrics are highly intertwined.
3. Analyze the data. Look at the analytics platform you installed to either view or download data to explore your identified metrics. Synthesize this information to identify its implications for your business or product.

Collecting self-reported information from people about their characteristics, thoughts, feelings, perceptions, behaviors, and attitudes

Surveys are a way to collect information from a larger sample of participants. Because of their ability to collect large amounts of data in a short amount of time, they are an efficient tool. Unlike other self-report methods such as interviews, surveys enable the collection of attitudinal information from a broader and more diverse range of participants. However, they often do not enable researchers to answer questions about why they answered a certain way and ask follow-up questions. Popular platforms for deploying surveys range in sophistication with Google Forms and Microsoft Forms providing relatively simple functionality, but available to all email account holders, to Survey Monkey, which provides a medium amount of functionality, and Qualtrics, which is highly sophisticated and customizable.

Steps for Conducting a Survey:

1. Define your overarching research question.
2. Develop the survey. Define key topics you hope to explore in your survey to break the survey into sections. Within each topic, identify tangible survey questions and intended formats (i.e. radio buttons, sliders, rating buttons, dropdowns, matrices, etc.) to ask respondents. Avoid too many free response questions as they lead to survey fatigue and open-ended questions, which will make data analysis difficult later on.
3. Recruit survey respondents. Many online platforms such as Amazon Mechanical Turk or Survey Monkey's built-in recruiting platform can assist in helping you screen and identify participants that fit your criteria.
4. Deploy the survey. Set the survey to be open and collecting responses for a set amount of time.
5. Analyze the survey data. Once the survey is closed, analyze the survey data. Online tools provide initial visualizations of the data, but data can also be downloaded for further analysis and examination of statistical significance in Excel, Tableau, Python, SPSS, or R.
6. Interpret survey results. What did you learn from your survey? What insights can you include?