Hiring for Open Mindedness in Product Managers: 3 Techniques to Try
Background
Over the course of my career, I’ve interviewed hundreds of product managers. But as I started getting deeper into researching decision making, judgement, and bias, I realized there was a hole in my interview process. One of the key traits of those who demonstrate excellent judgement is the ability to stay open and react appropriately to new information, no matter how much it conflicts with the existing norms and direction. I had some interview questions and exercises that would partially address it, but I was not doing anything deliberate to probe for this crucial skill.
Staying Open-Minded Is Critical
In the book Noise by Kahneman, Sibony, and Sustein, there is a passage on the correlation between those who demonstrate an ability to stay open-minded and consistent excellent judgement. The key is that people tend to fall in to one of two categories of how they approach judgement-related tasks; those who “go with their gut” and tend to stick with it regardless, and those who want to think deeply about the task. If you want to read more on this, check out the Cognitive Reflection Test. The punchline is that research suggests lower error rates for those who stay open to new information, look to disconfirm their own beliefs, and act on what they find.
This also is consistent with a fairly famous Bezos quote about some of the smarter people he’s observed change their minds a lot. I don’t necessarily agree with everything Amazon’s ever done, but I do respect the management system and culture Bezos put in place and the repeated results it produces in completely new industries.
3 Potential Interview Techniques to Get at Open-Mindedness
Product managers have to make countless decisions on a daily basis, and most interview processes don’t focus enough on predictors of decision quality and associated traits. Here are some thoughts on potential ways to probe for open-mindedness in an interview:
Make your own Cognitive Reflection Test
Most of the original Cognitive Reflection Test sample questions can be summarized as “they look like they have a straightforward answer but actually do not” and all tend to be mathematical. You certainly could frame an interview question that matches the spirit of the CRT but styled for your particular business and / or the job of product management.Introduce New Information During Assignment Briefing
I typically have candidates do an assignment during the interview process. What I have not done is give them new information during the case presentation that would strongly contradict prior background data given in the brief. It would probably take a little more forethought to get right and have the desired effect but could be really worth it. Given that the candidate has invested significant time in a presentation with conclusions based on data that points towards conclusion A, seeing if they have the ability to absorb, accept, and react appropriately to new data that points towards conclusion B (and partially negating the work they did) would be a real in the moment test of both good judgement and some of the key personality traits.“Tell me about a time when”
I’m fairly (in)famous for asking somewhat odd “tell me about a time when…” questions to try to get at particular personality traits. I think it would be a natural extension to add a few that try to get at times people have had to change course based on new data. Tell me about a time when you proved yourself wrong? Where did the contradictory data come from (e.g. did you find it)? In general I love “tell me about a time when” questions because asking several starts to paint a picture of the candidate’s level of self-awareness, reflection, and learning from prior events.
Wrap Up
I’m excited to start testing some of these new concepts in upcoming interviews and see what produces the best results. If you are not currently hiring, I’d suggest analyzing your current team on this same dimension. Are people staying open to information? Are they proactively looking for data to prove themselves wrong? What are ways you can help coach and guide the product managers on your team to have excellent judgement?
