18F Engineering end of year assessment guide

18F, as a part of GSA, has a mature 18F only, performance management and recognition system. This includes an end-of-year performance assessment. 18F Engineering’s goals for the end-of-year assessment are to have engineers talk with their facilitator and supervisor about the past year and create actionable plans for career growth in the coming year. These reviews ideally build on a robust, granular, continuous feedback loop between an engineer, their facilitator, and other coworkers.

This guide is historical although still largely applicable, although there may be changes from current practices. Please submit a PR if you see an explanation or process that needs updating!

Assessments are based on an individual’s performance plan. GSA’s Office of Human Resources Management (OHRM) has a 18F only, set of appraisal materials applicable to all of GSA. These include a good guide on 18F only, Appraising Objectively and Fairly, and the requirement to evaluate performance on a 5 point scale. Performance plans have what GSA calls “critical elements.” Each “critical element” in the performance plan is rated, and there is also a summary rating. Some ratings can earn performance awards. You can find more detail, including what each rating number means, in the 18F only, Appraisal Rating System & Performance Awards document.

Desired outcomes

Actionable feedback

The number one desired outcome was actionable feedback for individuals. In order for a review to be valuable, the person being reviewed needs to finish the process with knowledge of and a plan for how to grow over the following year. At the end of the review period, every engineer should have a solid understanding of what to do next to continue to improve and deliver better value to their teams.

Fair rating

Individuals deserve fair ratings. These ratings should be calibrated both within Engineering and across 18F. The ratings and feedback should be aligned with the individual’s performance profile (as is required of course), and not biased towards individuals that the raters have interacted with more than others or those whose strengths align the most with their raters’ interests. Positive and constructive feedback should be balanced, and all feedback should be backed by evidence. Ideally the individual agrees with the rating they’re given.

Holistic review

Individuals should be rated not only on their technical abilities and contributions, but on their total contribution to their teams and 18F. 18F Engineering performance plans include critical elements for technical and team contributions, and of course are how folks are rated. This means the reviewers must seek to understand the individual's role on their teams: for example, were they doing any project management or design work in addition to committing code. The reviewer should seek feedback from the individual’s project leads and more experienced technical staff they’ve worked with. Additionally, if an individual has contributed to other areas of 18F through guild and working group contributions, the reviewer should gather feedback from leaders and stakeholders in that group.

Process

The person conducting the review, usually an Engineering Supervisor, completes these steps.

1. Have the engineer complete a self-review.
2. Gather feedback from reviewers (key stakeholders including project leads, technical leads, peers identified in step 1, etc)
3. Synthesize that feedback into a review.
4. Review reviews with the Director.
5. Meet with the engineer and their facilitator to deliver and discuss the review.

More details on each step:

1. Conduct self-reviews

Have each individual complete a self-review. This review should ask for:

1. Which projects and accomplishments they’ve worked on over the last year, including both major staffed projects and also side-projects, working groups, guilds, etc.
2. A self-assessment. This is optional, and valuable.
   • How do you feel you are doing in each critical element in your performance plan
   • What are your strengths, where you are doing well
   • What are your areas for improvement
   • What training and support needs do you have for the next year
3. Peers (in engineering, or elsewhere) that they’d like us to solicit feedback from (in addition to project leads, tech leads, guild leads, etc., which we’ll talk to by default).
4. Any specific areas they’d like feedback on.
5. Answers to these six “engineering climate” questions (the same as we asked in the 18F only, mid-year review plus one new one). Note that we take advantage of the annual review process to collect this information but it does not factor into individual ratings:
   • Do you feel you’re growing as an engineer?
   • Do you ever feel like you’re not treated fairly? Please explain.
   • Do you feel like you can raise work issues? Do you feel like your issues are actually heard?
   • On a scale from 1 to 5, how satisfied are you with your work (1 is “not satisfied”, 5 is “very satisfied”)?
   • On a scale from 1 to 5, how motivated are you to do your work (1 is “not motivated”, 5 is “very motivated”)?
   • On a scale from 1 to 5, how satisfied are you with the feedback you’ve received throughout this year?

The answers are sensitive, and should only be shared with other supervisors, and director if necessary.

2. Gather feedback from reviewers

Now that you have the self-review, you can collect feedback from reviewers.

First, identify reviewers. The engineer’s facilitator can help with this. They should be the key stakeholders in the work the person has done: project leads, technical leads, working group and guild leads, and peers the person nominated above. You should identify at least 3-5 people, but if there are too many it’s OK to prioritize and pare the list down. Be sure to include people who can speak to project work throughout the fiscal year.

Send a request for feedback to each of the reviewers you identified above. This should ask for:

1. An assessment against each critical element in their performance plan (as for #2, above).
2. A request for feedback on the specific areas the individual identified in #4 above.
3. A free-form general feedback area.

Please remind reviewers to aim to be as specific as possible — ask for specific examples to back up any feedback they give. Assure reviewers that their responses will be synthesized and anonymized before they’re presented to the person under review (so we’re not going to say “well, so-and-so thinks you stink at Go!”)

If necessary, you can arrange short (15-30 minute) meetings with a reviewer to go over their feedback and gather specific examples. Whether you need to do this will depend on how good and specific written feedback is. Arranging a meeting can be a good method of chasing a response: it takes less time for the reviewer to tell you feedback than for them to write it up.

3. Synthesize feedback and draft a written review

Now you’ve got a bunch of feedback, so the next step is to synthesize it down into a coherent written review.

First, you’ll want to review all that feedback along with the person’s facilitator (if applicable). You’re looking for patterns, commonalities, or feedback that aligns with what you or the facilitator have personally observed. You’re looking to distill down all that feedback into a review for each critical element in the person’s performance profile. Include as many specific examples of behavior as possible — good feedback is specific and actionable (for a deeper treatment if this idea, see this article on The Situation-Impact-Feedback Tool).

It’s important to be especially cognizant of unconscious bias here. Remember that people are especially susceptible to assume that underrepresented people in tech — women, people of color, etc. — are less qualified than their white male counterparts. One way to correct for this bias is to make sure that reviews are concrete and specific (as above). Another way is to be cognizant of the kinds of reviews that are especially applied to underrepresented minorities. For example, the word “abrasive” is far more common in reviews of women than of men. Other similar kinds of phrases to watch out for are “tone”, “aggressive”, “judgemental”, “too nice”, and so on. In general do not write a review that seems like it speaks to the person’s character. Reviews should be focused on outcomes and project work.

GSA has 18F only, guidance on writing objective and fair reviews that’s worth reviewing as you start to write your review.

You’ll need to provide a score (1-5) for each critical element in the person’s performance profile. The performance plan has criteria for each score; these plans are available in CHRIS. You should also review the overall guidance on the 18F only, rating system scores. You’ll also need to give the engineer a single overall score (1-5), which is calculated from the critical element scores.

4. Review your drafts with the Director

Before finalizing the reviews, meet with the Director of Engineering and review your reviews. They’ll check that your reviews are fair and supported by sufficient evidence. They will check for bias, patterns, and consistency across Engineering. There is no forced ranking or forced distribution of ratings.

5. “Ship” the review, and go over it with the engineer

Now that the review is done, you need to file the “paperwork” and deliver the review.

First, create the review in CHRIS. Then export a PDF version and send it to the engineer at least a day in advance of meeting with them to deliver the review.

Then, meet with the engineer (and their facilitator, if applicable) to discuss the review. You’ll discuss the rating and possible performance award if applicable. This is also a good opportunity to deliver feedback that didn’t make it into the written review — especially feedback on areas that the engineer requested way back in step 1. Talk about ways the engineer can address specific feedback (and how we can support them). This can feed into a plan for next year, so take notes to help with follow up and evolution of next year’s plan.

Finally, discuss their answers to the “engineering climate” questions. There may have been details they didn’t want to provide in the survey version.

Lastly, the engineer will need to sign the review. You’ll store those PDFs in a Google Drive folder (Talent will send us the link), Talent’s tracking spreadsheet, and CHRIS.

Continuous feedback

Our intention for this process is that it is part of a regular, granular feedback system. Engineers deserve frequent feedback on their performance. Our structure, where engineers will have an established relationship with their facilitator and supervisor, allows us to ensure that feedback happens. Providing regular feedback makes the annual review process a simple extension of regular feedback, as it is a culmination of the year’s progress.