For Product Managers
Protecting Your Judgement: A Product Managers's Guide to Using AI Without Losing What Matters
When you paste user research into ChatGPT or let Notion AI summarise customer interviews, the AI finds patterns but misses the moment a customer's voice changed. When you feed your prioritisation criteria into an AI tool, it ranks features efficiently but cannot tell you what the data never captured. Your job is to keep your own judgement sharp while using AI to move faster.
These are suggestions. Your situation will differ. Use what is useful.
Read the Raw Research Before Asking AI to Summarise It
The customer empathy you need comes from hearing the contradiction in someone's words, seeing which problems they mention first, and noticing what they assume is already solved. If you send your Dovetail recordings straight to an AI summariser, you get themes and sentiment scores but lose the texture that changes your roadmap. Read or listen to at least one complete research session before you ask any tool to extract insights. This is not slower. It is the difference between building for what users say they want and building for what they actually need.
- ›Before using Dovetail AI to summarise, watch or read three full customer conversations yourself
- ›Keep a separate note of what surprised you or contradicted your assumptions before you run the AI analysis
- ›When the AI summary conflicts with your own notes, spend time understanding why instead of trusting the AI's pattern
Make Your Prioritisation Logic Visible Before You Hand It to AI
Prioritisation frameworks are where your product instinct lives. When you feed weighted criteria into Jira AI or Claude and get back a ranked backlog, the ranking looks objective but you have lost visibility into the trade-offs you actually made. Write out your prioritisation thinking in plain language first. Then use AI to check your work or to apply consistent scoring across a large backlog, not to replace the thinking. The AI should surface which items break your own rules, not decide what your rules should be.
- ›Document your prioritisation criteria and their weights in a shared doc before you use any AI tool
- ›Ask Claude to show you which backlog items don't fit your stated criteria instead of asking it to rank everything
- ›Flag items where the AI ranking contradicts your intuition and trace back to understand whether your criteria are wrong or incomplete
Use AI to Scale Research Synthesis, Not to Replace It
Your job includes holding the full picture of who your users are across multiple research methods and time periods. ChatGPT can help you find patterns across many user interviews or pull relevant quotes quickly. It cannot tell you whether a new finding contradicts something you learned six months ago or whether a trend is real or just loud. Use AI tools to do the work that is repetitive and time consuming, such as tagging interview transcripts in Dovetail or extracting feature requests from support tickets. Keep the work of interpretation and connection to your roadmap for yourself.
- ›Ask Dovetail AI to tag themes but create the tag definitions yourself based on your product model first
- ›Use ChatGPT to find supporting quotes for a hypothesis you have already formed, not to generate hypotheses from scratch
- ›When you find a potential insight in AI output, verify it against your source data instead of assuming the pattern is real
Protect the Direct Customer Contact That Builds Your Instinct
The best product decisions come from pattern recognition built over time through direct contact with users. When AI tools make research feel faster, it is tempting to do it more often but at more distance. You might have more data points summarised by AI but less time actually talking to customers. Block time each month to do customer research yourself, even when you have research specialists. This keeps your instinct calibrated and prevents you from building for the user that AI thinks exists rather than the user you have actually met.
- ›Schedule at least one customer call per month that you conduct yourself, not one that gets summarised for you
- ›Attend user research sessions in person or on video instead of reading the transcript first
- ›When you disagree with what an AI tool found, your next step is a customer conversation to test your instinct, not a longer prompt
Surface What the Data Did Not Capture in Your Roadmap Decisions
AI tools work with what is in front of them. They cannot tell you that your user research missed an entire customer segment because you recruited from your existing user base. They cannot flag that your usage data does not capture the problem customers gave up on solving before they abandoned your product. When you make a roadmap decision based on AI analysis, write down what data you did not collect and what customer behaviour you could not measure. Share this with your team. This practice stops you from mistaking clean data for complete data and keeps you honest about the limits of what you know.
- ›Create a section in your roadmap doc titled 'What We Did Not Measure' and update it when you release features
- ›When an AI tool says a feature is low priority, ask yourself which customer segment might have skipped the research because they stopped using your product
- ›Use Claude to help you design research that would test the opposite of what your current data suggests
Key principles
- 1.Read the raw research yourself before you ask AI to summarise it, because customer empathy comes from the parts of the data that are hard to quantify.
- 2.Your prioritisation logic is your product instinct, so document it clearly before you hand it to an AI tool to apply or check.
- 3.Use AI to scale the repetitive work of synthesis and tagging, but keep interpretation and connection to strategy for yourself.
- 4.Maintain direct customer contact each month so your instinct stays calibrated and you build for real users, not for the user that AI infers.
- 5.Explicitly name what your data did not capture when you make roadmap decisions, so you stay aware of the limits of what you know.
Key reminders
- When Dovetail AI or Claude finds a pattern you did not expect, your first step is to check whether the pattern is real by going back to source data, not to assume the AI has discovered something you missed
- Use prioritisation tools in Jira to show you items that contradict your stated criteria, not to discover what your criteria should be
- Ask ChatGPT to play the role of a customer who would disagree with your roadmap to test whether you have missed a user segment
- Keep a quarterly practice of comparing your product assumptions from your roadmap to what you actually learned from customers in the last three months
- When you feel uncertain about a product decision, that is often a signal to do customer research yourself rather than to ask AI to analyse existing data more thoroughly