40 Questions Operations Managers Should Ask Before Trusting AI
AI tools like SAP AI and Salesforce Einstein can spot patterns in data that humans miss, but they cannot know what happens on the floor when a shift gets cancelled or a process change breaks trust with your team. The right questions protect your operational judgement and catch the recommendations that will create more problems than they solve.
These are suggestions. Use the ones that fit your situation.
Questions About Process Improvement Recommendations
1When SAP AI recommends removing a quality check step to reduce cycle time, does the recommendation include data on the cost of failures caught by that step in the past year?
2If Copilot suggests consolidating two teams' workflows into one, have you asked the team leads whether the current separation exists because of dependencies that the data does not show?
3Does the AI recommendation account for the training time and temporary productivity loss your team will experience during the changeover?
4When the tool recommends a new process, can you trace the data sources it used, or is it drawing conclusions from incomplete transaction logs?
5Has anyone from the floor level who actually runs the current process reviewed the recommendation, or did it go from AI output to your decision?
6If the improvement was implemented as-is, which team member would be blamed when it fails, and does that person have input into whether it should happen?
7What does the recommendation assume about staff availability, skill level, or willingness to change that may not be true for your team?
8Has the AI measured the time it takes to fix problems caused by process changes, or only the time saved by the change itself?
9Does the recommendation depend on perfect data entry or perfect adherence to the new process, both of which are unrealistic?
10If you implement this recommendation and it creates friction between your team and another department, who will resolve that problem?
Questions About Performance Dashboards and Metrics
11When Tableau AI builds a new performance dashboard, what metrics did it exclude, and why did it choose the ones it included?
12Does the dashboard measure what actually matters for your business, or does it measure what is easiest to collect from your systems?
13If your best-performing team member's work does not show up well on this dashboard, is the problem with the person or with the metric?
14Has anyone tested whether hitting the dashboard targets actually correlates with the outcomes your organisation cares about?
15When the dashboard flags a problem, can you trace back to the original transaction or decision, or only to a number that looks anomalous?
16Does the dashboard account for seasonal variation, one-off events, or other factors that make month-to-month comparison misleading?
17If you manage team behaviour to optimise dashboard performance, would that be the right thing for your customers or your organisation?
18How long after something goes wrong on the floor does the dashboard show it, and are you relying on the dashboard instead of direct communication from your team?
19When Einstein suggests a resource reallocation based on performance data, does it know about the difficult client or pending deadline that explains this person's current numbers?
20Does the dashboard measure individual performance in a way that discourages the cross-team collaboration you need to stay competitive?
Questions About Scheduling and Resource Allocation
21When SAP AI recommends a new shift pattern, does it account for the fact that some of your experienced staff cannot work those hours?
22If the algorithm assigns tasks based on historical speed, does it know that your fastest performer is about to go on leave?
23Has the scheduling recommendation been checked against the informal knowledge your team lead has about who works well together and who does not?
24When Copilot suggests stretching one team member to cover a gap, what happens to the work they currently do well, and who decided that trade-off was acceptable?
25Does the resource allocation model account for the fact that some people are better at training new staff or mentoring juniors, even if they are not the fastest at the core task?
26If the algorithm recommends moving someone to a new role or team to balance utilisation, have you considered the cost of losing their institutional knowledge in their current area?
27When the tool optimises for cost, does it measure the cost of turnover, complaints, or safety incidents caused by overwork or demotion?
28Has anyone asked the person being reallocated whether they agree it is the right move, or are you presenting it as a system decision?
29Does the scheduling tool know which of your clients or projects have higher error tolerance and which ones need your most experienced people?
30If the new schedule creates bottlenecks in a different part of the operation, is that accounted for in the recommendation, or only in your department's metrics?
Questions About Your Own Operational Judgement
31What warning sign have you caught before data showed it as a problem, and does the AI system have any way to flag that type of thing?
32When was the last time you overrode an AI recommendation, and what did you know that the data did not capture?
33If you stopped checking the details and started trusting the dashboards completely, what would you stop noticing?
34Which of your best operational decisions came from hunches based on experience rather than from the data you had at the time?
35When ChatGPT or Copilot writes a communication to your team about a change, how does it sound compared to how you would explain it, and does that difference matter?
36Are you spending less time on the floor and more time in the system because the AI tools make it feel like you do not need to check in person?
37What have your team members told you is wrong that never made it into a formal report or system?
38When you make a decision based on AI output but without deeper reasoning, are you ready to defend it if it goes wrong?
39Does relying on AI recommendations for day-to-day decisions mean you would struggle to make good decisions if the system went down?
40Have you noticed yourself becoming more confident in decisions because an AI tool backed them up, even though your actual knowledge has not changed?
How to use these questions
Before you act on an AI recommendation, ask someone who actually does the work whether they spotted the same problem. If they did not, ask them why.
Metrics that look good on a dashboard often hide the things that matter most. Keep one operational measure that the system does not track and check it weekly.
When an AI tool optimises for one metric, it is always sacrificing something else. Identify what that something else is before you approve the change.
Your instinct that something is wrong on the floor is data too. Write it down, investigate it, and do not dismiss it because the dashboard looks normal.
Ask the AI to show you the edge cases and exceptions it encountered, not just the main pattern. That is where you will find out what it does not understand about your operation.