Skip to main content
U.S. flag

An official website of the United States government

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Discover

Build a greater understanding of your problem and the people it impacts.

Cognitive walkthrough

What

An evaluation method in which people work through a set of representative tasks and ask questions about the task as they go.

Why

To get quick and early feedback on whether a design solution is easy for a new or infrequent user to learn, and why it is or isn’t easy. This method is useful for catching big issues at any stage in the design process when you don’t have access to real users, but it is not a substitute for user evaluation.

Time required

30 minutes to one hour per person

How to do it

  1. Identify specific traits for new or infrequent users of a design solution.
  2. Develop a set of representative tasks that emphasize new use or infrequent use.
  3. Designate a member of the design team to play the role of a user. That person will use the traits you’ve identified to participate in a moderated usability testing session. (The traits can overlap.)
  4. Ask the user to accomplish their goal using a printed or interactive design. As they go, ask what they would attempt to do next or how they would learn.
  5. Don’t lead the user through the task, but encourage them to stay focused on what they’re trying to accomplish.
  6. Pay attention to expected outcomes and how quickly/easily participants are able to pick up a task.
  7. Analyze the walkthrough results to highlight where the user struggled and what needs improvement.

Considerations for use in government

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation (e.g., not a survey) that a cognitive walkthrough entails, 5 CFR 1320.3(h)3.

If you are not working with government employees, you will need to observe standard precautions for archiving personally identifiable information.

18F

Contextual inquiry

What

The product team unobtrusively observes participants at work, with their permission, then asks questions.

Why

To learn how and why users do what they do; to discover needs and attitudes that might not emerge in an interview to map how tools, digital and otherwise, interact during complex activities.

Time required

1-2 hours per user

How to do it

  1. With permission from a supervisor and from the participant, schedule a time to watch a typical work activity and record data.
  2. While observing, ask the participant to act normally. Pretend you're a student learning how to do the job. Ask questions to help you understand what the person is doing and why.
  3. At the end of the session, explain what you have learned and check for errors.
  4. Immediately after, write up your notes.

Example

A pair of 18F team members visited two Department of Labor/Wage Hour Division investigators as they interviewed home health care workers who were subject to unpaid overtime and other infractions. Since it was a sensitive subject, the 18F team did not question the health care workers directly, but instead asked the investigators clarifying questions in private. 18F staff also made sure that photos did not include faces.

Additional resources

Considerations for use in government

No PRA implications, if done properly. Contextual interviews should be non-standardized, conversational, and based on observation. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

For internal folks, get permission from the right level of management. If participants could be under union agreements, contact the agency's labor relations team.

18F

Design studio

What

An illustration-based way to facilitate communication (and brainstorming) between a project team and stakeholders.

Why

To create a shared understanding and appreciation of design problems confronting the project team.

Time required

3–4 hours

How to do it

  1. Invite between six and 12 participants: stakeholders, users, and team members who need to build a shared understanding. Before the meeting, share applicable research, user personas (unless users will be present), and the design prompt for the exercise.
  2. Bring drawing materials. At the start of the meeting, review the design prompt and research you shared.
  3. Distribute drawing materials. Ask participants to individually sketch concepts that address the prompt. Remind them that anyone can draw and artistic accuracy is not the goal of the exercise. 15–20 minutes.
  4. Have participants present their ideas to one another in groups of three and solicit critiques.
  5. Ask the groups to create a design that combines the best aspects of members’ individual contributions.
  6. Regroup as a whole. Have each group of three present their ideas to everyone. Discuss.
  7. After the meeting, note areas of consistent agreement or disagreement. Incorporate areas of consensus into design recommendations and areas of contention into a research plan.

Example from 18F

Considerations for use in government

No PRA implications. If conducted with nine or fewer members of the public, the PRA does not apply, 5 CFR 1320.5(c)4. If participants are employees, the PRA does not apply.

18F

Dot voting

What

A simple voting exercise to identify a group's collective priorities.

Why

To reach a consensus on priorities of subjective, qualitative data with a group of people. This is especially helpful with larger groups of stakeholders and groups with high risk of disagreement.

Time required

15 minutes

How to do it

  1. Bring plenty of sticky notes and colored stickers to the meeting.
  2. Gather everyone on the product team and anyone with a stake in the product.
  3. Quickly review the project's goals and the conclusions of any prior user research.
  4. Ask team members to take five minutes to write important features or user needs on sticky notes. (One feature per sticky note.)
  5. After five minutes, ask participants to put their stickies on a board. If there are many sticky notes, ask participants to put their features next to similar ones. Remove exact duplicates.
  6. Give participants three to five colored stickers and instruct them to place their stickers on features they feel are most important to meeting the project's goals and user needs. If the group is recently formed, or for other reasons has a low level of established trust and equity, consider a set-up where people can vote privately, e.g. on paper. You’ll need a few extra minutes to compile and display.
  7. Identify the features with the largest number of stickers (votes). Have the team review the voting for whether familiarity bias is resulting in enshrining the status quo.

Considerations for use in government

No PRA implications: dot voting falls under “direct observation”, which is explicitly exempt from the PRA, 5 CFR 1320(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Five whys

What

An iterative process for identifying the root cause of a problem by posing the question “Why?” at least five times to help separate symptoms from causes.

Why

To identify the root cause(s) of an issue or problem.

Time required

Less than 1 hour

How to do it

Select a particular issue or problem from your user research to investigate further. This could be the most commonly occurring problem or a problem that has been prioritized by the team. Ask why the problem occurred and write down an answer. Repeat this process another four times, building off of the previous response each time to drill down to a root cause. As you probe, make sure you remain sensistive to the emotional response of the interviewee. Sometimes asking why multiple times can cause the interviewee to feel frustrated or defensive if they don’t feel as if they are being heard. See example below:

Starting problem: “We didn’t meet our goal for public feedback during the open comment period.”

  1. Why?
    “Not enough people submitted comments.”
  2. Why?
    “Not enough people made it to the comment submission form.”
  3. Why?
    “The comment submission form was hard to find.”
  4. Why?
    “The link to the comment submission form was buried on the page.”
  5. Why?
    “We didn’t formulate and publish a call to action to submit comments.”

After getting to a root cause, frame or reframe your problem solving approach to address it (e.g., “how might we create a call to action for comment submission?”).

Note: You may ask “why” more or less than five times during this process. The purpose of this exercise is to help identify what is the root cause. Ask “why” as many times as needed to get to what you think the root cause is, while keeping the mental cost of the interviewee in mind.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Heuristic evaluation

What

A quick way to find common, large usability problems on a website.

Why

To quickly identify common design problems that make websites hard to use without conducting more involved user research.

Time required

1–2 hours

How to do it

  1. Recruit a group of three to five people familiar with evaluation methods. These people are not necessarily designers, but are familiar with common usability best practices. They are usually not users.
  2. Ask each person to individually create a list of “heuristics” or general usability best practices. Examples of heuristics from Nielsen’s “10 Usability Heuristics for User Interface Design” include:
    1. The website should keep users informed about what is going on, through appropriate feedback within reasonable time.
    2. The system should speak the user’s language, with words, phrases and concepts familiar to the user, rather than system-oriented terms.
  3. Consider what hazards the service or product might hold for its users, and include heuristics to evaluate whether the site protects against possible harms.
  4. Ask each person to evaluate the website against their list and write down possible problems.
  5. After individual evaluations, gather people to discuss what they found and prioritize potential problems.

Additional resources

Considerations for use in government

No PRA Implications, as heuristic evaluations usually include a small number of evaluators. If conducted with nine or fewer members of the public, the PRA does not apply, 5 CFR 1320.5(c)4. If participants are employees, the PRA does not apply. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Hopes and fears

What

An exercise that quickly surfaces a group’s hopes and fears for the future

Why

To establish a baseline understanding of a group’s expectations and concerns about a project and to give each person an opportunity to voice their perspective

Time required

30–60 mins

How to do it

  1. Ahead of the session, establish what you want to elicit hopes and fears about. For example, you could ask participants to focus on the whole project or that day’s workshop.
  2. At the beginning of the session, create two columns labeled “Hopes” and “Fears” on a white board or large sticky pad. (In a remote setting, you can do this online using collaboration software such as Mural or Google Docs)
  3. Ask participants to take 1-2 mins to write down their hopes on sticky notes (one hope per sticky note).
  4. Invite participants to come up one at a time and add their “hopes” sticky notes to the board and say more about what they wrote. Have participants group their sticky notes as they add them to the board to illustrate emerging themes.
  5. Repeat steps 3 and 4 with fears.

This format can be adapted to include other categories. For example, asking participants to write down skills and experiences can help contextualize each person’s place in the group.

Example

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

KJ method

What

A facilitated exercise in which participants list their individual priorities onto cards, collect them as a group, organize them by relationship, and establish group priorities through individual voting.

Why

To reach a consensus on priorities of subjective, qualitative data with a group of people. This is especially helpful with larger groups of stakeholders and groups with high risk of disagreement.

Time required

1–2 hours

How to do it

  1. Gather four or more participants for 90 minutes. Provide sticky notes and markers.
  2. Create a focused question about the project’s needs and select a facilitator to run the exercise.
  3. Give participants five minutes to write at least three responses to the question, each on its own note.
  4. Give participants 15 minutes to put their answers on the wall, read everyone else’s, and make additions. Have participants cluster similar answers without discussion.
  5. Ask participants to write names for each cluster on their own - this is mandatory. They may also split clusters.
  6. Put each name on the wall by its cluster. Exclude word-for-word duplicates.
  7. Reiterate the question and have each person rank their three most important clusters. Visually tally points.
  8. Combine duplicates and their points if the entire group agrees they’re identical. Three or four groups usually rank higher than the rest - these are the priorities for the question.

Example from 18F

18F conducted this exercise with 20 Federal Election Commission staff members to define priorities around conflicting requests. We used this method to get data from staff (not the decision makers) about what they saw as the most pressing needs. We synthesized and presented the data back to the decision makers.

Considerations for use in government

At 18F, KJ participants are almost always federal employees. If there is any chance your KJ workshop could include participants who are not federal employees, consult OMB guidance on the Paperwork Reduction Act and the Privacy Act. Your agency’s Office of General Counsel, and perhaps OIRA desk officers, also can ensure you are following the laws and regulations applicable to federal agencies.

18F

Lean coffee

What

A format for running a meeting without a predefined agenda

Why

To give everyone equal opportunity to surface ideas and vote on agenda topics, allowing meeting attendees to be co-owners in the meeting agenda.

Time required

Flexible

How to do it

  1. Give meeting participants two minutes to write what they would like to talk about on sticky notes (one idea per sticky note)
  2. Have meeting participants review the topics generated and dot vote on the topics they are most interested in
  3. Decide how much time will be spent talking about each topic
  4. Start with the topic that got the most votes
  5. At the end of the allotted time, have meeting participants vote:
    • Thumbs up: Continue talking about the topic for a shorter set amount of time
    • Thumbs down: Move to the next topic

Example from 18F

At 18F, Lean coffee is often used to facilitate community of practice meetings and team meetings when the objective is to provide a forum for the meeting attendees to raise issues that are of interest to them. This method provides structure to these meetings and ensures topics are democratically selected for conversation.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Stakeholder and user interviews

What

A wide-spanning set of semi-structured interviews with living experts who have an interest in a project’s success, including stakeholders and users.

Why

To build consensus about the problem statement and research objectives.

Time required

1–2 hours per interviewee

How to do it

  1. Create a guide for yourself of some topics you’d like to ask about, and some specific questions as a back up. Questions will often concern the individual’s role, the organization, the individuals’ needs, and metrics for success of the project. Consider how the interview could harm the participant, and adjust your questions to avoid those hazards. For example, might your questions trigger thoughts of painful experiences?
  2. Sit down one-on-one with the participant, or two-on-one with a note-taker or joint interviewer, in a focused environment. Introduce yourself. Explain the premise for the interview as far as you can without biasing their responses.
  3. Follow the conversation where the participant takes it. They will focus on their priorities and interests. Be comfortable with silences, which allow the participant to elaborate. To keep from getting entirely off course, use your interview guide to make sure you cover what you need to. Ask lots of “why is that” and “how do you do that” questions. Consider asking if there are ways the service or product could cause harm to its users if not done carefully, or what assumptions are being made (leaving it ambiguous as to whether we’re referring to our questions or in the service/product).
  4. If there are other products they use or your product doesn’t have constraints imposed by prior work, observe the stakeholders using a competing product and consider a comparative analysis.

Additional resources

Considerations for use in government

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Stakeholder influence mapping

What

A visual representation of stakeholders — the people who are involved — and their potential influence and impact on a project or service system in comparison to one another.

Why

To uncover and describe power dynamics — the often-unspoken balances of influence and control — that can impact project outcomes. Stakeholder influence mapping also helps us prioritize which stakeholders to engage with and how, and informs our communication and engagement approach.

Time required

~1 hour

How to do it

  1. Gather the team and at least one crucial stakeholder familiar with their organization and how it works from both a technical and an interpersonal point of view.
  2. Using a whiteboard (or virtual collaboration tool), divide the board into a grid with 4 sections. Label the x-axis influence and the y-axis interest.
  3. List out stakeholders together. Write down names of people, groups, communities, or organizations that your work may impact, and organize them into the four quadrants based on your understanding of their relative influence and interest.
  4. Look at each quadrant to sort who to engage with and how. For stakeholders that are both interested and influential, collaborate closely with them. For stakeholders that are either influential or interested, keep them informed. For stakeholders that are neither influential nor interested, allow them to drive their own involvement.
  5. If your map reveals power dynamics that route around policy, consider whether the information poses personal or professional risk to any stakeholders. Avoid possible harm by sharing this map with only the people who need to understand it, and consider the consequences of those you do share with — or share an edited version. Be sure to review and update the map as you understand the situation better.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

System map

What

A diagram that shows how different factors and forces affect a given system. This is unlike journey maps or service blueprints, which visualize specific services.

Why

Mapping complex patterns can help us build consensus and identify opportunities. Maps can also be used to onboard new team members and inform product decisions.

Time required

4-8 hours

How to do it

  1. Choose a sketching app or gather pen and paper.
  2. Identify the problem you want to focus on and the specific questions you want to visualize in a system. For example, what health food options currently exist in a community?
  3. Identify the people affected by this problem. Invite them to engage in the following steps.
  4. Identify data sources to create your system map. Your sources may come from engaging with people most affected by the problem, looking at secondary data, or conducting other qualitative research.
  5. Draw arrows between the different parts of your system to identify how they’re connected.
  6. Reflect on specific areas to examine more closely. What questions come up? What gaps do you see?
  7. Share the map with others and invite them to add to it. Include people affected by the problem, key stakeholders, and subject matter experts.
  8. Update the map as you learn more about the problem.

Additional resources

Considerations for use in government

No PRA implications. Even when users are present, the PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. If you are not working with government employees, you will need to observe standard precautions for archiving personally identifiable information.

18F

Decide

Elaborate on research from your Discovery phase.

Affinity mapping

What

A way of finding themes in collections of ideas, quotes, or observations.

Why

To draw out insights from qualitative data quickly and collaboratively.

Time required

1 hour

How to do it

  1. Record ideas, quotes, or observations from interviews, contextual inquiry, or other sources of research on sticky notes.
  2. Place the sticky notes on a white board (in no particular arrangement). Move the sticky notes into related groups.
  3. Use larger notes (or white board markers, if you're using a white board), to write titles or catch phrases for each group.

Considerations for use in government

No PRA implications. This method may use data gathered from members of the public, but does not require their involvement.

18F

Archetypes

What

Archetypes are visual documents that represent a group of users based on their shared goals, needs, attitudes, behaviors, and/or pain points.

Why

Unlike demographic data that can unintentionally introduce creative blocks or bias, archetypes can help better understand product or service users who share the same behaviors and core desires.

Time required

30 mins-1 hour per archetype

How to do it

  1. Gather findings from research activities such as contextual inquiry or stakeholder interviews, user surveys, and analysis of customer data and organize them in a way that’s easy to review.
  2. Analyze and affinity map your research findings for patterns. Note frequently observed goals, motivations, behaviors, pain points, and potential harms (e.g. lack of consent, physical danger, being retraumatized). These patterns help form the foundation of each archetype.
  3. Create sets of user archetypes based on how you believe people will use your solution and the journeys they are on. These archetypes typically get titles, for example, “the marketing specialist” and will include narratives that describe motivations in a relatable way.

Example at 18F

Tips for remote participation

Archetypes can be created virtually using any variety of digital whiteboards or word-processing tools.

Additional resources

Applied in government

No PRA implications. No information is collected from members of the public.

18F

Coding qualitative data

What

Using flexible coding tools for sorting spreadsheets or databases to analyze qualitative data.

Why

Quickly organize research to help you see patterns and relationships. To help make qualitative data actionable and to give it an even playing ground with quantitative data.

Time required

3–4 hours or more depending on the amount of data collected.

How to do it

  1. Gather and organize qualitative data within your tool of choice.
  2. Sort or tag the data based on themes and categories. Organizing the content from your research using filters or tags.
  3. Analyze the data based on the identified themes. By sorting and resorting your findings, you'll be able to quickly identify patterns, relationships, and insights, which are essential for synthesis.
  4. Share your findings with your team(s) to determine the next steps.

Example from 18F

When doing feedback sessions for a prototype of a new dot gov registrar, we used qualitative data analysis to help sift through the gathered feedback to identify key areas for the next of design and development.

Tips for remote participation

Common for multiple team members to break off to take a pass at sorting data for themes and come back together to share and discuss findings.

Considerations for use in government

When using qualitative data, remove any personally identifiable information (PII) from your documents before sharing.

18F

Comparative analysis

What

A detailed review of existing experiences provided either by direct competitors or by related agencies or services.

Why

To identify competitors’ solutions that excel, are lacking, or are missing critical design elements. Comparative analysis can give you a competitive edge by identifying opportunities, gaps in other services, and potential design patterns to adopt or avoid.

Time required

1–2 hours to analyze and write an evaluation about each competitor.

How to do it

  1. Identify a list of services that would be either direct or related competitors to your service. Pare the list down to four or five.
  2. Establish which criteria or heuristics you will use to evaluate each competing service.
  3. Break down the analysis of each selected competitor into specific focal areas for evaluation. For example, how relevant are search results?
  4. Use a spreadsheet to capture the evaluation and determine how the targeted services and agencies perform based on the identified heuristics.
  5. Present the analysis, which should showcase areas of opportunities that you can take advantage of and design patterns you might adopt or avoid.

Example from TTS

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Content audit

What

A listing and analysis of all the content on an existing website (including pages, files, videos, audio or other data) that your users might reasonably encounter.

Why

To identify content that needs to be revised in new versions of a website. Content audits can also help you identify who is responsible for content, how often it should be updated, and what role a particular piece of content plays for users.

Time required

3-8 hours

How to do it

  1. Identify a specific user need or user question that you’d like to address.

  2. Create an inventory of content on your website. Navigate through the site from the home page and note the following about every piece of content. (For repeated items like blog posts, consider capturing just a sample.)

    1. Title used in the site’s navigation for that page
    2. Title displayed on the page or item itself
    3. URL
    4. Parent page
  3. Identify the main entry points for the user need you’re addressing. This could be external marketing, the homepage, a microsite, or another page.

  4. From each entry point, trace the pages and tasks a user moves through until they address their need.

  5. For every piece of content they might come across on that task flow, note:

    1. Author(s): who wrote or created the page
    2. Content owner(s): who ensures its credibility
    3. How often or when it was last updated
    4. Comments: qualitative assessment of what to change to better address your identified user need

Example

Additional resources

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Design hypothesis

What

Framing your work as a hypothesis means no longer just thinking about the thing you’re making or building, but paying more attention to whether that work is achieving your intended goals and outcomes.

This in turn means thinking about your work as a series of experiments you do with your users to learn if you’re on the right path. Instead of asking “Did we ship the shopping cart feature?” you ask: “Did we make it easier and simpler for our customers to buy from us?”

Why

When done collaboratively, hypothesis-building is powerful at getting a team on the same page about what it’s doing and why. It also allows the team to be flexible — if one approach doesn’t result in the outcome you expected, you have implicit permission to change course and try something else.

Time required

1-2 hours

How to do it

  1. As a team, identify and make explicit the problem you’re trying to solve. What goals or needs aren’t being met? What measurable criteria would indicate progress toward those goals?

  2. As a team, write out the hypothesis for the work you want to do to address the problem(s) you’re trying to solve. You may want to write broad hypotheses at the outset of a project and more specific hypotheses each sprint.

    Here’s a common way to structure your hypothesis:

    We believe that doing/building/creating [this] for [this user] will result in [this outcome].

    We’ll know we’re right when we see [this metric/signal].

  3. Once you’ve formulated your hypothesis, consider the following harm prompt to help the team think about and guard against potential unintended consequences of your work.

    But, this could be harmful for [this user] if [this outcome happens].

  4. Identify a user touchpoint that will allow you to test your hypothesis, such as external marketing, the homepage, a microsite, or something else. Test your hypothesis. If you learn something unexpected, refine your hypothesis, test again, and continue to work incrementally towards your goals.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Design principles

What

Written statements, generally in the form of imperatives like “Earn people’s trust,” that serve as guiding lights during decision-making.

Why

To give the team and the stakeholders a shared point of reference when negotiating next steps. Good design principles are specific to the project, not general truths, and should help teams say “no” to otherwise interesting proposals or generate ideas when they’re stuck.

Time required

1 week, plus occasional refresher meetings

How to do it

  1. Using internal documents and kickoff activities, gather terms or concepts that seem significant to project goals and organizational culture.
  2. Using existing research, list terms or concepts that seem particularly important to customers or user groups.
  3. Cluster similar terms and concepts together on a whiteboard or other writing space open to everyone in the project. Name the clusters.
  4. Ask the team and stakeholders if they would like to add, change, or edit any concepts or groups.
  5. From the groups on the board, create three to five final principles. Using evidence from partner or user research, write one to two sentences in support of each principle.
  6. Share the principles in a place accessible to the team throughout the project, and refer to them often while making decisions.

Examples

Considerations for use in government

No PRA implications. Generally, no information is collected from members of the public. Even when stakeholders are members of the public, the PRA explicitly exempts direct observation and non-standardized conversation (e.g., not a survey), 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Interface audit

What

A listing and analysis of all the components, design patterns, and interface features of an existing website (including typography, color, graphics/illustration/icons)

Why

To identify components that need to be revised in new versions of a website to create consistency and fill gaps. Interface audits can also help you establish and document a design system for a website.

Time required

Depends on scope of audit (how many pages, how many contributors, etc)

How to do it

An interface audit can be conducted by an individual or in a group setting. Either way, the steps are as follows:

  1. Identify the website and take screenshots of all the pages you want to audit
  2. Create a checklist of aspects you want to audit on each page—for example typography, header and body copy styles, use of color, buttons, icons, etc.
  3. For each page, take notes on each aspect on your checklist.
  4. Once all pages have been audited, compare notes and identify inconsistencies (e.g., headers are inconsistently formatted, sometimes bolded, sometimes italicized).
  5. Decide how to resolve any inconsistencies by choosing one of the existing approaches found on the site (e.g., make all headers bold) or designing a new solution (e.g., make all headers a different color).

Note: It’s helpful to involve developers, who will be able to advise on the feasibility of potential solutions.

Additional resources

Applied in government research

No PRA implications. No information is collected from members of the public.

18F

Journey mapping

What

A visualization of the major interactions shaping a user’s experience of a product or service.

Why

To provide design teams with a bird’s-eye view of a service that helps them see the sequence of interactions that make up a user’s experience including the complexity, successes, pain points, and emotions users experience from the earliest phases of researching a product or service all the way through adoption.

Time required

4–12 hours

How to do it

  1. Document the elements of the project’s design context. This includes:
    • People involved and their related goals
    • Their behaviors in pursuit of their goals
    • Information, devices, and services that support their behaviors
    • Important moments in how they experience a service or major decisions they make
    • The emotions associated with these moments or decisions
    • Potential harms (e.g. financial penalties, anxiety, damaged reputation) associated with these moments or decisions
  2. Visualize the order in which people exhibit behaviors, use information, make decisions, and feel emotions. Group elements into a table of “phases” related to the personal narrative of each persona. Identify where personas share contextual components.
  3. Discuss the map with stakeholders. Point out insights it offers. Use these insights to establish design principles. Think about how to collapse or accelerate a customer’s journey through the various phases. Incorporate this information into the project’s scope.

Additional resources

Considerations for use in government

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Mental modeling

What

A simple reference model that correlates existing and potential interfaces with user behaviors.

Why

To help designers anticipate how design decisions might facilitate future behaviors.

Time required

1–2 hours

How to do it

  1. Create one three-columned table per persona. Label the columns “Past,” “Present behavior,” and “Future.”
  2. In the middle column (Present behavior), list current user behaviors and pain points broadly related to the project, one per row.
  3. In the left-hand column (Past), list the products, services, features, and/or interfaces that the user encounters as they go about what’s listed in the Present behavior column.
  4. In the right-hand column (Future), list possible products, services, features, and/or interface elements that in the future might change behaviors and pain points in the Present behavior column.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Personas

What

User archetypes based on conversations with real people.

Why

To ground design in reality by forcing us to consider the goals, behaviors, and pain points of the people affected by our design decisions. Unlike marketing personas based on demographics or marketability, design personas describe how someone accomplishes goals.

Time required

2–3 hours

How to do it

  1. Gather research from earlier activities like contextual inquiry or stakeholder interviews in a way that’s easy to review. You can create placeholder personas without research to teach user-centered thinking, but because they’re effectively stereotypes, avoid using them for implementable design decisions.
  2. Create a set of user archetypes based on how you believe people will use your solution. These typically get titles (for example, “data administrators” rather than “those who submit data”). Review the archetypes with “who questions:” Who is included? Who is being overlooked? Who is deciding whom to include?
  3. Analyze your records for patterns as they relate to user archetypes. Specifically note frequently observed goals, motivations, behaviors, and pain points, and potential harms (e.g. lack of consent, physical danger, being retraumatized).
  4. Pair recurring goals, behaviors, pain points, and potential harms with archetypes. Give each archetype a name and a fictional account of their day. Add a photo of someone who fits the description, but ideally not an image of someone you’ve actually interviewed and who may be recognized.
  5. Link your personas to the research that inspired them. This is useful when researchers are interested in challenging the way a persona stereotypes a user.

Example

Additional resources

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Service blueprint

What

A visual representation of the start-to-finish experience of both using and supporting the delivery of a service, including staff interactions and user experience.

Why

To clarify relationships between intertwined systems and processes. By communicating the full complexity of a service, service blueprints help teams find opportunities for improvement.

Time required

4-12 hours

How to do it

  1. Gather information on the service through desk research and interviews with users, frontline staff, and support staff.
  2. Create a diagram with four rows:
    • User steps: the primary action someone takes when interacting with the service
    • Frontstage actions: the online and offline interactions that users have with the service, including people, places where interactions occur, and physical or virtual objects that users interact with, like forms
    • Backstage actions: activities in the systems and processes that support the frontstage experience, but are not visible to users
    • Support processes: activities executed by the rest of the organization or external partners — such as ongoing data management or software licensing — that don’t fall into the other rows
  3. Map the flow of each user interaction through the service. Note critical points and interactions in delivering the service, as well as any opportunities to handle interactions more efficiently or that would result in a better user experience.

Examples

Considerations for use in government

No PRA implications. No information is collected directly from members of the public.

18F

Site mapping

What

A comprehensive rendering of how a website’s pages relate to one another.

Why

To audit an existing website by assessing its structure and content. Site maps also help you plan and organize the contents of a new website prior to wireframing and building it.

Time required

2–3 hours

How to do it

  1. List each page of a website or section.
  2. Take a screenshot of each page. Create a thumbnail for each screenshot.
  3. Print the thumbnails on individual pages if completing this exercise in person. Remote teams can use a shared whiteboard tool. Arrange the page thumbnails into a hierarchical diagram. Focus on the logical relationships between pages. If you’re evaluating an existing website, focus more on these relationships than on the URL structure. If some pages function as sub-pages to another, the site map should reflect that.
  4. Use the diagram to guide choices about things like information architecture and URL structures.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Storyboarding

What

A visual sequence of a specific use case or scenario, coupled with a narrative.

Why

To visualize interactions and relationships that might exist between a user and a solution in the context of the user’s full experience.

Time required

1–2 days depending on the complexity of the scenario(s)

How to do it

  1. Gather any documents that describe the different use cases or scenarios in which users will interact with your service.
  2. Sketch scenes that visually depict a user interacting with the service, including as much context as possible. For example: Are they on the move? Where are they? What else is in their environment?
  3. Annotate each scene with a description of what the user is attempting to do. Describe what general feeling or experience the team wants the user to have.
  4. Review this storyboard with the product team and stakeholders for feedback. Iterate until the storyboard represents a shared vision of the scenario and progression of scenes.
  5. Create a polished version of the storyboard if you plan to use it for future work or in other external contexts.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Style tiles

What

A design document that contains various fonts, colors, and UI elements that communicate the visual brand direction for a website or application.

Why

To establish a common visual language between the design team and stakeholders. It also acts as a collaboration artifact that both the design team and stakeholders can use to contribute to the final design direction.

Time required

1–2 days depending on how many rounds of feedback the team offers

How to do it

  1. Gather all the feedback and information that was provided during the initial kickoff of the project.
  2. Distill the information into different directions a solution could take. Label these directions based on what kinds of interactions and brand identity they represent.
  3. Create the appropriate number of style tiles based on the defined directions, which establish the specific visual language for the different directions.
  4. Gather stakeholder feedback. Iterate on the style tiles, eventually getting down to a single style tile which will be the established visual language for the project going forward.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Task flow analysis

What

A step-by-step analysis of how a user will interact with a system in order to reach a goal. This analysis is documented in a diagram that traces a user’s possible paths through sequences of tasks and decision points in pursuit of their goal. The tasks and decision points should represent steps taken by the user, as well as steps taken by the system.

Why

To validate a design team’s understanding of users’ goals, common scenarios, and tasks, and to illustrate in a solution-agnostic way the overall flow of tasks through which a user progresses to accomplish a goal. Task flow diagrams also help surface obstacles in the way of users achieving their goal.

Time required

2-3 hours per user goal

How to do it

  1. Based on user research, identify target users’ goals that need to be analyzed.
  2. For each goal, identify common scenarios and the tasks and decisions that the user or system will perform in each scenario. Don’t assume you and your stakeholders share the same understanding of the tasks. The idea is to make the flow of tasks explicit in the diagram, so that you can check your understanding by walking through the diagram with users (steps 4 & 5).
  3. Produce a diagram that includes each task and decision point that a user might encounter on their way toward their goal. While there are several diagrammatic languages that can be used to produce task flow diagrams, the basic look is a flow chart of boxes for tasks and decision points and arrows showing directionality and dependencies among tasks. The diagram should cover the common scenarios identified in step 2.
  4. Present the diagram to a subject matter expert who knows the task(s) well enough to check for accuracy.
  5. In collaboration with users and/or subject matter experts, annotate the task flow diagram to pinpoint areas of interest, risk, or potential frustration.

Additional resources

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

User scenarios

What

A method for telling a story about a user’s interaction with your product, service, or website, focusing on the what, how, and why.

Why

To communicate a design idea by telling a story about a specific interaction for a specific user. Through creating user scenarios, you’ll identify what the user’s motivations are for using your product, service, or website, as well as their expectations and goals. User scenarios help teams consider both how the same user’s needs might vary depending on their context and how a diverse group of users in the same scenario might have different needs. By constructing user scenarios, you can help the team answer questions about how accessible, inclusive, and adaptive your product, service, or website is.

Time required

1-3 hours

How to do it

  1. Determine a few personas or user groups to focus on. Consider what scenario(s) might be the most critical for that user, including scenarios in which users face limited accessibility.

  2. For each user, list out their goals, motivations, and the context/environment in which they interact with your product, service, or website.

  3. Put the details you came up with in step 2 into a story format that includes the following information:

    • who they are (persona or user group)
    • why they are using your site (motivations)
    • where they are (context)
    • what they need to do (their goal)
    • how they go about accomplishing the goal (tasks)

    Keep in mind, the more realistic details you add, the richer and more useful your story becomes for helping to understand your user’s behaviors.

  4. Share the user scenarios that you’ve written with the user group (and other relevant team members) for validation, feedback, and refinement.

  5. Examine your product, service, or website in light of these user scenarios and identify opportunities to make adjustments that would improve users’ experiences.

Additional resources

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

User stories

What

A short, plain-language description of a product feature or requirement written from the perspective of the user.

Why

To frame the problem we’re solving around the needs of the people who will use the product, rather than the limitations of the technology or the program or agency overseeing it. User stories help us break down a large development effort into smaller, more manageable slices so that we can track progress, plan sprints, and prioritize work around user impact. They also encourage us to talk about features in plain language and make the discussions (and solutions) more accessible.

Time required

1-2 hours per epic

How to do it

  1. Start by defining the project’s epics—typically the larger, high-level “themes” or bodies of work that make up the initiative.
  2. Break down each epic into pieces of work that can be independently estimated, developed, and tested. These will become your user stories.
  3. For each piece of work you identified, write a story from the user’s perspective of who they are, what they want to do, and why. User stories typically follow this template:

    As a < type of user >, I want < some goal > so that
    < some outcome >.

    Note: Only make the type of user as specific as you need to. If the piece of work is needed by all your users, it’s ok to say “as a system user.” That way when it is specific (“as a first-time applicant”) it’s clear that this is a group with particular needs.

  4. Consider possible harms to the user, the system, or external to the system by adding a second part of the sentence:

    …while avoiding < potential negative impacts > to
    < the user, the system, or external to the system >.

  5. User stories can be the basis for tickets—a tool to manage and track work for the solution you’re building. These tickets should include the story, a brief description, and a list of acceptance criteria.

Additional resources

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Make

Create a testable solution.

Design pattern library

What

A collection of UI elements used frequently across a design system, consisting of the base patterns and helpful information about how to use them.

Why

To aid in designing a solution that uses UI elements consistently. Maintaining a set of approved, reusable patterns makes it easier to produce new features or make updates to the current solution.

Time required

1–2 hours per pattern; ongoing maintenance.

How to do it

  1. Start identifying common components as early as possible, ideally while you and the team are creating new design elements. These common pieces form the patterns that you will create guidelines for. Specify the components that make up each UI pattern and note possible constraints or restrictions.
  2. Describe or visualize how someone will use the pattern and how it should respond to the user. (For example: how a button renders on load, hover, and click.) Provide any data as to why it is good for the end user.
  3. Include any code or snippets that front end developers can use to implement the pattern.
  4. Show examples of how the same pattern could work in different solutions.
  5. Publish the design pattern library in an open, accessible space where the product team can use and extend it. (Common implementations of a design pattern library are in a wiki or brand style guide.)

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Prototyping

What

A rudimentary version, either static or functional, of something that exhibits realistic form and function.

Why

To enable direct examination of a design concept’s viability with a number of other methods such as usability testing or a cognitive walkthrough. Static prototypes (often paper) are helpful for gaining feedback on users’ intentions and various design elements. Functional prototypes (often coded) are helpful for observing how users interact with the product.

Time required

4 hours

How to do it

  1. Create a rudimentary version of your product. It can be static or functional. Think in the same way you would about a wireframe: demonstrate structure and relationships among different elements, but don’t worry about stylized elements.
  2. Give the prototype to the user and observe their interactions without instruction.
  3. After this observation, ask them to perform a specific task.
  4. Ask clarifying questions about why they do what they do. Let the user’s behavior guide the questions you ask. It can be helpful to have them narrate their thought process as they go along.
  5. Iterate! Prototypes should be quick and painless to create, and even more quick and painless to discard.

Considerations for use in government

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Wireframing

What

A simple visual representation of a product or service interface.

Why

To prioritize substance and relationships over decoration as you begin defining the solution. Wireframing also gives designers a great opportunity to start asking developers early questions about feasibility and structure.

Time required

1-3 hours

How to do it

  1. Build preliminary blueprints that show structure, placement, and hierarchy for your product. Steer clear of font choices, color, or other elements that would distract both the researcher and the reviewer. Lightweight designs are conceptually easier to reconfigure. A few helpful tools for building wireframes are OmniGraffle and Balsamiq, which purposefully keep the wireframe looking like rough sketches.
  2. Use this opportunity to start listing what UX/UI patterns you will need.
  3. Review your wireframes with specific user scenarios and personas in mind. Can users accomplish their task with the wireframe you are sketching out?
  4. Use the wireframes to get the team’s feedback on feasibility and structure.

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

Validate

Test a design hypothesis.

Card sorting

What

A categorization exercise in which participants divide concepts into different groups based on their understanding of those concepts.

Why

To gain insights from users about how to organize content in an intuitive way.

Time required

15–30 minutes per user

How to do it

There are two types of card sorting: open and closed. Most card sorts are performed with one user at a time, but you can also do the exercise with groups of two to three people.

Open card sort

  1. Give users a collection of content represented on cards.
  2. Ask users to separate the cards into whatever categories make sense to them.
  3. Ask users to label those categories.
  4. Ask users to tell you why they grouped the cards and labeled the categories as they did.

Closed card sort

  1. Give users a collection of content represented on cards.
  2. Ask users to separate the cards into a list of categories you have predefined.
  3. Ask users to tell you why they assigned cards to the categories they did.

Example from 18F

Additional resources

Considerations for use in government

No PRA implications if done as directly moderated sessions. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3.

18F

Multivariate testing

What

A test of variations to multiple sections or features of a page to see which combination of variants has the greatest effect. Different from an A/B test, which tests variation to just one section or feature.

Why

To incorporate different contexts, channels, or user types into addressing a user need. Situating a call to action, content section, or feature set differently can help you build a more effective whole solution from a set of partial solutions.

Time required

2–5 days of effort, 1–4 weeks elapsed through the testing period

How to do it

  1. Identify the call to action, content section, or feature that needs to be improved to increase conversion rates or user engagement.
  2. Develop a list of possible issues that may be hurting conversion rates or engagement. Specify in advance what you are optimizing for (possibly through design hypothesis).
  3. Design several solutions that aim to address the issues listed. Each solution should attempt to address every issue by using a unique combination of variants so each solution can be compared fairly.
  4. Use a web analytics tool that supports multivariate testing, such as Google Website Optimizer or Visual Website Optimizer, to set up the testing environment. Conduct the test for long enough to produce statistically significant results.
  5. Analyze the testing results to determine which solution produced the best conversion or engagement rates. Review the other solutions, as well, to see if there is information worth examining in with future studies.

Considerations for use in government

No PRA implications. No one asks the users questions, so the PRA does not apply. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Success metrics

What

How to measure whether your design is successful in achieving the intended outcomes.

Why

No matter what you’re designing (software, a service, content, etc.), you need to define what your design should enable people to do and how you’ll measure whether it performs well.

Time required

2-3 hours to brainstorm and select metrics.
Length of prototype run to collect data.

How to do it

  1. Define the outcomes you expect when your design performs well. Invite stakeholders to brainstorm. Your outcomes may be based on your design hypotheses or user needs statements.
  2. List ways to measure whether your design is achieving expected outcomes. Consider quantitative and qualitative metrics:
    1. Quantitative metrics are numerical indicators (e.g. time on task, click-through rates).
    2. Qualitative metrics capture subjective feedback and insights (e.g. usability ratings, user satisfaction).
  3. Determine how you will collect data for each metric. This may involve user interviews, usability testing, surveys, and/or analytics. Choose metrics that are effective and measurable.
  4. Plan who will be responsible for data collection and how often metrics will be reviewed.
  5. Establish benchmarks for each metric. Benchmarks show whether the design meets peoples’ needs or should be further refined.
  6. Test your design, and evaluate success based on your measurement plan.
  7. Analyze the results against your benchmarks. Identify strengths and areas of improvement. Iterate on the design prototype. Refine metrics if necessary.

Additional resources

Considerations for use in government

Surveys of more than 9 people require PRA approval. Learn more at PRA.Digital.gov. Direct observation and non-standardized conversations like semi-structured interviews) are not subject to PRA.

Your prototype may not be the only effort being tested. People may be overwhelmed by the amount of data they’re asked to provide. Finding metrics that can be automatically collected can help relieve this burden.

18F

Usability testing

What

Observing users as they attempt to use a product or service while thinking out loud.

Why

To better understand how intuitive the team’s design is, and how adaptable it is to meeting user needs.

Time required

30 minutes to 1 hour per test

How to do it

  1. Pick what you’ll test. Choose something, such as a sketch, prototype, or even a "competitor’s product" that might help users accomplish their goals.
  2. Plan the test. Align your team on the scenarios the test will focus on, which users should participate (and how you’ll recruit them), and which team members will moderate and observe. Prepare a usability test script.
  3. Recruit users and inform their consent. Provide a way for potential participants to sign up for the test. Pass along to participants an agreement explaining what participation will entail. Clarify any logistical expectations, such as screen sharing, and how you’ll share links or files of whatever it is you’re testing.
  4. Run the tests. Moderators should verbally confirm with the participant that it’s okay to record the test, ask participants to think outloud, and guide the participant through the session. Observers should contribute to a rolling issues log and relay any in-session questions to the moderator, refraining from interrupting the session from the participant’s point of view. Engage your team in a post-interview debrief after each test.
  5. Discuss the results. Schedule a collaborative synthesis meeting to discuss issues you observed, and any questions these tests raise concerning user needs. Conclude the meeting by determining how the team will use what it learned in service of future design decisions. main

Examples from 18F

Additional resources

Considerations for use in government

No PRA implications. First, any given usability test should involve nine or fewer users. Additionally, the PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. It also specifically excludes tests of knowledge or aptitude, 5 CFR 1320.3(h)7, which is essentially what a usability test tests. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Visual preference testing

What

A method that allows potential users to review and provide feedback on a solution’s visual direction.

Why

To align the established branding guidelines and attributes of a solution with the way end users view the overall brand and emotional feel.

Time required

4-12 hours for style tiles. 30 minutes per participant to get feedback.

How to do it

  1. Create iterations of a style tiles or other asset that represent directions a final visual design may follow. If branding guidelines or attributes don’t exist, establish them with stakeholders beforehand.
  2. Interview participants about their reactions.
    • Ask questions as objectively as possible.
    • Align questions with the branding guidelines and attributes your project must incorporate.
    • As far as possible, allow participants to provide their feedback unmoderated or at the end of your research.
  3. Compare the results of your research with the agency’s published branding guidelines and attributes.
  4. Publish the results to the complete product team and decide which direction will guide future design efforts.

Considerations for use in government

No PRA implications. The PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3. See the methods for Recruiting and Privacy for more tips on taking input from the public.

18F

Fundamentals

Foundational methods for practicing design research.

Compensation

What

Offering usability test or user research participants compensation to encourage participation and to thank them for their time.

Why

Compensating participants for sharing their time and lived experience with your team often results in a more diverse, representative set of participants. Without compensation, you often end up recruiting people with a strong intrinsic interest in your website. These people may not have the same needs and experiences as a less interested pool of users. With compensation, you can encourage less interested, more representative people to participate.

Time required

N/A

How to do it

  1. Figure out what’s legal and appropriate. Consult your agency’s Office of General Counsel on options for providing compensation to encourage participation in usability testing, consistent with your agency’s authorities. The options will depend upon your agency’s authorities and the specific facts.
  2. Consider contracting for a recruiting service to help you get an effective research pool.
  3. If compensation is allowed, clearly communicate when and how participants will receive compensation. In the emails, postings, or other materials you use to recruit your participants, describe the compensation and how participants will receive it (via mail, pick up at an office, etc.). This is particularly important for “remote” research.

Example

Considerations for use in government

No PRA implications. Even when users are present, the PRA explicitly exempts direct observation and non-standardized conversation, 5 CFR 1320.3(h)3.

If you are not working with government employees, you will need to observe standard precautions for archiving personally identifiable information.

18F

Privacy

What

Designers potentially might work with many different categories of information, across a number of different contexts. You have an obligation to steward information in a way that respects privacy.

Why

Designers have an obligation to respect and protect privacy. People will not honestly participate in design processes, nor make use of products and services, they do not trust.

Time required

N/A

How to do it

  1. Familiarize yourself with the Fair Information Practice Principles, a set of precepts at the heart of the U.S. Privacy Act of 1974.
  2. Consult your organization’s privacy office, which may include your general counsel, if you plan to substantially make use of information that could potentially identify specific individuals.
  3. Inform and collect the voluntary consent of anyone who participates in moderated design research. Ensure that all unmoderated forms of research (for example, web analytics) are covered by an easy-to-access privacy policy.
  4. Pay special attention to all categories of information used throughout the design process. Note contexts in which it’s not okay to share certain categories of information.

Additional resources

Considerations for use in government

The government’s use of information about people is subject to a number of laws and policies, including: the Privacy Act of 1974, the Federal Information Security Management Act of 2002, and the eGovernment Act of 2002.

Conduct a Privacy Threshold Analysis in collaboration your agency’s privacy office whenever a design calls for the creation of a new data store (for example, a database).

Ensure all collections of personally identifiable information (PII) are accompanied by a Privacy Act Notice. See, for example, GSA’s Privacy Act Notice for Design Research.

18F

Recruiting

What

Identifying and gathering people to interview or who will test your product.

Why

Recruiting people who represent your core user group is a critical and oft-overlooked part of research. Time spent with the right people using the wrong methods is better than time spent with people who aren’t your core users while using the right methods.

Time required

1–2 weeks for 5–10 participants

How to do it

Seek out people who

  • Are trying to use the thing you are working on right at that very moment.
  • Recently tried to use the thing you are working on.
  • Used the thing you are working on less recently.
  • Have used something like what you are working on, and are likely to use what you are working on.

Reach them through

  • Relevant community organizations.
  • Impromptu requests in or near the relevant environment.
  • Your personal and professional network.
  • The new or existing website.
  • Existing mailing lists.

Additional resources

Considerations for use in government

No PRA implications. No information is collected from members of the public.

18F

18F Methods

An official website of the GSA’s Technology Transformation Services

Looking for U.S. government information and services?
Visit USA.gov