Appendix

Verbal interview question bank

Below are a set of sample questions to use in verbal interviews during acquisition to elicit additional clarity about a contractor's technical approach in their proposal.

Note: verbal interviews are not the same as oral presentations under FAR Part 15. Oral presentations allow a contractor to amend or change their proposal. Verbal interviews do not permit a contractor to amend or change their proposal; instead they provide an opportunity to ask clarifying questions on the technical approach portion of their proposal as written.

Technical approach

Engineering

Talk about your process for determining which software and programming language to use as your development team iteratively builds software products. We're also interested in hearing the rationale for the initial software and programming languages in your technical response.
What is your technology stack of choice for this project, and why? Which technology stacks does this particular team have the most experience with? What other stacks/technologies are the team experienced with?
Describe your technical development and collaboration process. Please specify your approach to version control, testing (and test-driven development), accessibility, continuous integration, and continuous deployment.
Discuss the technical decisions you've made in your proposal, and what outstanding questions those decisions raised about this project. In particular, how do you plan to address the needs of this product's multiple user groups?
Please discuss your approach to test-driven development and continuous integration and deployment.
How will you approach technical oversight? How would you track the standards described in the QASP?
How would you identify deep problems within a codebase? After identifying those issues, how would you address and reduce technical debt? What types of refactoring strategies would you consider?
What do you anticipate as the largest risks in backend development?
How do you intend to address data security needs and requirements?
Tell us about a time you jumped in in the middle of a development effort. What challenges did you face? How did you overcome them? How do you envision integrating yourself within the existing development effort?
Please describe your technical lead's experience with [programming language].
Tell me about a system you built on top of some infrastructure- or platform-as-a-service.
Tell me about an infrastructure problem that you helped solve (for example: slow application performance, unexpected downtime, a security breach; etc). What was the problem, and how did you solve it?

Collaboration

How would you ensure good communication with all team partners?
Walk us through how you see the designers and developers interacting as you build the product.
Talk about how you'd like the [agency] to be involved as you design and build the product.
What activities do you plan on engaging in to ensure a strong, collaborative process with regard to teamwork?
Your agile development approach looks to have a number of handoffs between different parts of the team. Can you tell us how you can ensure that the various members of the design/dev team will be able to stay on the same page during implementation?
How do you typically communicate your findings and strategic recommendations to a client? How do you frame things when the findings might challenge some of your partners' assumptions?
Have you ever worked with a remote/distributed team before?
- Yes: What tools and/or mechanisms did you use to help promote open dialogue and foster communication between yourself and your teammates? How did you overcome any communication challenges and barriers?
- No: What challenges do you anticipate? What do you think you'll need to succeed?

Staffing

Talk about how you would quickly staff your design/dev team if awarded the contract.
Tell us more about the different team roles you envision for this project.

Research and design

How have you incorporated changes based on user research?
What do you see as the paramount user-centric feature(s) that users will interact with in the system?
How would you design a usability test for an iteration of this product? What participants would you recruit? What tasks would you use? How would you analyze the results?
How would you go about determining the visual feel and content tone of the project?
How would you bring the full team and stakeholders into the research and synthesis process?
How have you incorporated changes based on user research in other projects?
Please discuss your experience with usability testing.
Describe a time when you uncovered research results that disproved the team's core assumptions. What was the situation? What did you do? What did you learn?

Product and strategy

Talk about what you see as the three most important risks for this project and how you will help the [agency] to mitigate them.
What do you need from the government product owner to make this project succeed?
How will you develop a product vision and prioritize features? How do you envision ongoing feature prioritization working?
Tell us about a project you led that was particularly challenging or complex. How did you approach it? How did it work out?
What if an agency wanted something and you don't think they need it?

Iterative development

Describe the agile project management practices and tools you will use for estimating, planning, managing risk, team collaboration, and communicating status. Why would you use these?
How will you keep developers, designers, and researchers engaged in building out a user story without lengthening the sprint or reverting to a "waterfall" development process?
Talk through how you see the development team interacting with the [agency's] product owner to ensure reasonably sized sprints for the development team.
How would you alert the government product owner if the team encounters tasks that require more work than originally anticipated that can't be completed in the current sprint?
Tell me about your experience with agile or other iterative development styles. How does practicing agile affect the technical choices you make?

Based on the answers, also dig in on anything that is pertinent on a given project, or that seems odd in a given proposal, for example:

Tell us more about what you envision will happen during the month-long system definition kickoff.
Talk through your rationale for why you think PHP will be the right fit for what [agency] will need to build.
Explain your choice of datastore technology in your technical approach. Detail both benefits and problems.
What do you see as the main challenges of building and testing an API, and what steps do you recommend to overcome those challenges?
What are some of the compromises you think will need to be made in meeting performance budgets in mobile devices? How do you intend to address those compromises? What does mobile responsiveness mean to your team?
What is your risk management strategy regarding PII?
How do you intend to secure data-in-transit using a FIPS 140-2 compliant encryption module?
How will your technical approach account for low bandwidth environments with limited or old technology?
What was your thinking behind choosing labor categories that did not directly correspond to the labor requirements?
Can you please describe your experience with Pay.gov?

Federal Field Guide