If you don’t care about my table-setting (I know I can be long-winded), jump to the step-by-step guide.

I want to start this post with the major caveat that interviewing in tech is broken. Most of our interviewing practices are entrenched in systemic bias, don’t evaluate people fairly or on what we mean to evaluate, and require a lot of free labor that disproportionately disadvantages people with marginalized identities. This post isn’t about dismantling the broken tech interviewing system. I don’t have answers for that (yet). This post aims to work within the system and find ways to make 1 particular interview practice—take-home code challenges—more representative and fair for all.

This post is written from the perspective of someone who has the privilege in their company to make changes to the interviewing process. I know that not everyone has this privilege. If you do, I hope you’ll lend some of it after reading to make changes at your company. Let's give others entering the industry or switching jobs a positive, empowering experience instead of the gatekeeping one so many of us have had.

Have you ever been given a code challenge as part of the interview process at a company and realized that it wasn't at all aligned with the role you're applying for? Maybe you’re a frontend dev who has been given a challenge centered around database architecture. This is incredibly common and frustrating. When challenged, many companies respond with, “This is the prompt that everyone gets.” They choose to sacrifice meaningful assessment in favor of consistency of the problem. And they signal that either 1) they don’t know what role they’re hiring for or 2) they don’t care. In both cases, it’s probably a waste of your time to even attempt that challenge.

If you are an interviewer at this same company, you can’t get a sense if any given person will be successful in any given role. How could you when you are evaluating people for different roles with different expectations and responsibilities using the same prompt?

If you walk away with nothing else from this post, walk away with this. Your code challenge should be representative of the work the person will do regularly in the role they are applying for.

But Mercedes! Rewriting our code challenge is going to be so much work! First, yes it will be. But it will save you a lot of time and money. Hiring is expensive. If you can save time during the interview process because the interviewers can more accurately evaluate candidates and hire people who are better aligned with your expectations, you will decrease your cost per hire. Second, I have a step by step guide so I simplified a lot of it for you 🤗 You just have to follow the steps and do the work.

Step-by-step guide

Thanks for your patience during the long intro, there was a lot to unpack. Here is my step-by-step breakdown of creating fairer and more representative code challenges. While you’re going through this, it might be useful to do it with a pair or small team rather than solo. I used this process with my co-worker, Shamyle, and I’m excited to put our new challenges into practice.

1. Identify the specific role and expectations that you are writing the challenge for

We’re going to use the expectations of the job role to tailor the code challenge. If you don’t have defined expectations for the role you’re hiring for, stop here and go do that. It’s unfair to hire someone without expectations. How will you or they know if they’re successful? How will you fairly evaluate candidates? Set your expectations first, I’ll wait. This post will be here when you get back.

Career path or career ladder documents are my go-to resources for identifying role expectations. Another good one is the company’s evaluation criteria during performance review cycles. Find any other documents your company may have that defines expectations. Read the description(s) for the role that you’re hiring for. Pull out the technical expectations that it defines and create a list for yourself.

2. Identify characteristics you do and don’t like on other code challenges

Using your current challenge(s) and others that you have seen or taken, identify what things you like in a code challenge and what you don’t. This step is meant to catch all the little details outside of the technical requirements.

Think about what type of writing style and tone you like in code challenges. Think about how you prefer the requirements to be outlined. Consider how much narrative you enjoy.

Reflect on how you like a problem presented—in the context of work? the real-world? fantasy? etc. However, take care to be inclusive when considering context. Don’t place the problem within a context that some of your interviewers may not be familiar with, unless it’s core to the work you do or vertical that you’re in.

How prescriptive do you like your challenges? Do you prefer code challenges that are more open-ended? Is the challenge representative of the type of work you and your team do? What scope and time expectations do you prefer—30 min? 2 hours? 4 hours?

When Shamyle and I did this step for our company, some of the things we identified were:

What we like

• Stories/table-setting that anyone can relate to and doesn’t require special knowledge
• Including definitions or glossary
• Expanding the problem set as the role progresses in seniority
• Tech-stack agnostic problems

What we don’t like

• Problems not similar to our daily work, such as graph theory problems
• Video game examples

3. Identify goals that you have for your new code challenge

Now think about some goals that you have with rewriting your code challenge. This step sets your success criteria for this process. When you’ve finished writing the challenge, we’ll look at these goals to see if you achieved what you set out to do.

Think about why you are rewriting the challenge. What do you hope to change? What do you want your candidates to get out of the challenge? What do you want your interviewers to experience?

Here are some of the goals Shamyle and I identified:

• Leave our candidates feeling successful when they complete the challenge
• Make the challenges fun for interviewers and code reviewers too
• Give the candidates a sense for the kind of work we do
• Help candidates learn about our company

4. Identify skills that you want the challenge to allow you to evaluate

In step 1, you compiled a list of the technical expectations of the role. Now hone that list down to exactly what you want to evaluate in the challenge. Not all of the expectations you have for the role may be easy to evaluate or fit within the scope of a take-home code challenge. For example, deploying into an environment with complex networking requirements in addition to coding a feature or application might not make sense.

Be sure to think about both the code-related technical skills and the communication-related technical skills. You may want a candidate to demonstrate a firm understanding of the MVC pattern and also solid documentation skills. Or you may want your candidate to demonstrate strong unit testing skills but also be able to explain why they chose the unit testing library they did.

Here are a few of the skills that Shamyle and I identified for this step:

• Problem comprehension
• Proficiency of language and awareness of tools for chosen language
• Software testing
• Written technical communication

5. Identify common problems that you and your team solve at work

Next, spend some time to think about real problems that you and your team encounter in your projects. We’ll use this as the skeleton of the problem for the challenge.

If you work at a product company, what types of problems are your team tasked with solving? Are you on the performance optimization team? Does your team work a lot with security? Do you manage large amounts of data? Do you work with internal tooling? Is scale a challenge for you?

If you work for a consulting firm, what types of clients do you work with a lot? What problems do you find yourself solving on each project? Why do clients choose your company over others?

Shamyle and I work at a consulting firm. Our clients come to us with a lot of similar problems. Some of the common features we implement include:

• Scheduling
• Task Automation
• Reporting
• Integrating with 3rd-party systems

5b. (optional) Identify things around the office or about your company culture that set you apart

One of Shamyle’s and my goals was to help our candidates learn about the company. I’m including this as an optional step because you may not share that goal. But this was the most fun part of the process and helps make the challenge uniquely you.

Think about some of the unique perks you get working at your company. How do coworkers like to spend time together? What makes your company fun and different? What sets you apart from your competitors? Be careful that you don’t fall into traditional tech stereotypes of beer on tap and ping-pong tables as these can feel othering to people from marginalized communities.

When Shamyle and I considered these questions for our company, we came up with:

• We pair program a lot
• We eat a lot of tacos
• We have a lot of plants in the office
• We do a lot of puzzles together

6. Put it all together

Now choose one of your common problems from step 5 as the core of your code challenge prompt. Consider what requirements you need to define for the candidate to apply the skills from step 4. If you did step 5b, layer in one of these ideas as the table-setting for the problem. If you didn’t do step 5b, consider your answers to step 2 and work in the type of narrative or structure that you like.

Refine as you go, consistently checking in with your goals from step 3. Does your prompt and the potential solutions meet what you’ve outlined?

Make sure that the problem statement and requirements you put together are appropriately leveled for the role and expectations you’re interviewing for. For example, if you are hiring for an early-career position where extensive data modeling is not an expectation, make sure you are providing static data for the candidate. Depending on how prescriptive you wanted your prompt, outline assumptions appropriately or leave them open-ended.

7. Solicit feedback from your team!

Share the draft of your new challenge with team members. Ask about their thoughts on scope, relevance, and style. Does the challenge seem appropriately scoped and sized for your expectations of time the candidate should spend? Does the problem seem aligned with the work the candidate would be doing once they joined the team? Is the style and tone consistent with your company culture?

Iterate on your challenge and incorporate your team’s feedback until you’re happy and the prompt achieves your goals 🤗

8. Package it for distribution

Finally, make sure to add the administrative and logistic expectations to your prompt. How should the candidate package and submit? Are there any "must haves" or “don't dos” that the candidate needs to know? Outline the timeline for the process and next steps so your candidate is informed and knows what to expect. Set communication expectations for both your company and the candidate.

Conclusion

Using fair and representative code challenges can drastically alter your hiring process. Your candidates walk away from the process being better informed about what it’s like to work at your company and the type of work they would do. And your interviewers are better equipped to evaluate candidates accurately based on relevant skills and experience.

If you liked this breakdown of the process and want to try it yourself, check out the worksheet here.