Everyone around me seems to be talking about the home assignments that product managers are given during the interview process.
As a (past) recruiter, I love home assignments. As a (past) candidate, not so much.
I do believe, however, that home assignments are a critical tool in evaluating candidates for senior positions, and product management — even in a beginner’s role — is a senior position.
I also believe that when done right, it provides an opportunity for the candidates to shine, and not only a necessary evil.
Unfortunately, I see it done wrong many times. It is causing so much frustration that people are starting to protest against it.
Since I don’t want to lose this important tool, I wrote a quick guide on how to do it right. If you have anything to add — don’t be shy. That’s what comments are for. Let’s make this a productive discussion.
Rule #1: Don’t give assignments as the first step of the interview process
I mean it, seriously. Please don’t.
Assignments are taking a long time and effort from the candidate. Give them the minimal respect and do your screening process by investing your time as well. Give them the opportunity to meet you and consider whether or not they want to invest this time and effort in your company. The good product managers — the ones you are looking for — are interviewing you just as much as you are interviewing them. Respect them to gain their respect.
I personally believe that a home assignment should be the last step in the process (or one before last if you are big enough for the CEO/relevant executive to not be deeply involved in hiring). You should be able to assess your candidates in other ways before you make them work hard to help you to know them better.
I only give assignments to candidates I am highly impressed by, and I use the assignment as a final validation of what I think, not as a time-saver mechanism.
Rule #2: Understand what you want to test with the assignment, and build it accordingly
Much like in an A/B test, you want to be very clear on what is it that you need validation for.
Hint: it can’t be everything.
Hint #2: the assignment you plan needs to be connected to what you want to test.
For example, if you want to see how someone thinks (this is usually what I am looking for in the assignment), there is no point in asking them to design screens. Some companies even go as far as forbidding notes, asking for screens only. If you want to test their design skills — that’s a great test. The outcome should speak for itself (and even that is not always true, see rule #5).
But if you want to see how they are thinking about a problem — the final outcome has to be able to tell you that. If you want to see their thought process, the outcome should explain the process — so it should be a document or a presentation.
Rule #3: Use a neutral domain
Following the previous rule, unless you want to test domain expertise, use a product from another domain as your test case. Otherwise, you are adding pressure and bias where you don’t need to. Don’t use their current product domain either.
A good product manager should be able to understand new domains, and that happens all the time as part of our work.
I found throughout the years that a neutral domain also works best for me as a recruiter since it allows me to objectively assess the product skills and not be biased be domain expertise — mine or theirs.
Rule #4: Set very very clear expectations
Note that I didn’t say instructions. I said expectations, and I meant it. Expectations should include:
- How much time are you expecting them to invest in it (be honest, don’t say 1–2 hours just because it sounds good. You can fine-tune your answer by asking candidates how much time they really put into it eventually. And if your conclusion is that it should take 10 hours, find another assignment. It’s too long.)
- What should they do about the information they are missing (they will always miss information since you are asking them to do a product manager’s job for a company they don’t work in. I always tell the candidates that they can assume anything reasonable, as long as they state their assumptions clearly.)
- How will you evaluate the assignment
- What is most important for you to see, since time is limited (let’s admit it, doing it right always takes much longer than we give them)
Acknowledge that you know they will not be able to complete it the way they want to, that there are many things they can’t know coming from the outside, that the task is (deliberately) broad and blurry.
What you are asking for is not trivial, see it from the candidate’s perspective and make it easier where you can (without compromising what you are trying to test).
Rule #5: Invite them over to present and discuss their solution
An outcome of a product manager’s job rarely speaks for itself. The same goes for their home assignments. Let the candidate explain how they got there and why.
Since there is never a correct answer in product management, just a well-explained and thought-of answer, you can see “correct” outcomes which were achieved by coincidence, and “incorrect” outcomes which become right when explained.
Plus, you will learn so much from how the candidates present, how they answer questions or respond to objections — all of which are an important part of a product manager’s job.
Ideally, have a number of people in the room, from a variety of disciplines, who can help assess the outcome and discussion from multiple perspectives.
Product leaders: make decent use of the home assignments. It is a powerful tool but should be used with caution so that it doesn’t wear out.
Final word to candidates
I see many complaints about companies using candidates’ outcomes of home assignments to develop the real product. I haven’t seen a single company that does that, and I have seen many companies.
What I have I seen regarding this topic are things like home assignments about a feature that is already in development but not yet released, or about a feature that was considered and decided to be put off for now.
The fact that they could — theoretically — steal your ideas into the product, doesn’t mean that they are actually doing it.
Alternatively, if you see something which resembles your home assignment on the final product, it might be because you got — with partial information — to the solution that they thought of, and they were dealing with it day in and day out. It means that you are a great product manager, even if you didn’t get the job. My recommendation is to take the compliment and move on. Your dream job is waiting for you elsewhere.