40 Questions Cybersecurity Analysts Should Ask Before Trusting AI
When Darktrace flags a thousand anomalies or Splunk AI ranks alerts by confidence score, your next action depends on whether you understand why the tool made that call. The analysts who catch novel attacks are the ones who question the AI output before they act on it.
These are suggestions. Use the ones that fit your situation.
1When Microsoft Security Copilot assigns a severity score to an alert, can I see which three pieces of evidence carried the most weight in that score?
2Has this Darktrace model anomaly appeared in my organisation before, or is the tool showing me something that looks new only because the baseline changed?
3If I remove the most recent 30 days of data from Splunk AI's training set, does this alert still rank as high priority, or was it pattern-matching to something temporary?
4What is the false positive rate for this specific alert type in my environment, not the vendor's benchmark environment?
5Is the alert triggered by a behaviour that is actually suspicious, or by a behaviour that is simply rare in the training data?
6When CrowdStrike AI says this process is malicious, is it because the process itself is known-bad, or because it appeared in an unusual execution context?
7Could this alert be generated by legitimate tooling that my organisation uses but the AI model has not learned to recognise?
8Is the alert based on real-time detection or on retrospective log analysis, and does that change how I should prioritise my response?
9What would this alert look like if it were a false positive, and how different is it from the alert I am seeing now?
10Has the AI model been trained on my organisation's normal traffic patterns, or only on industry-wide baselines?
Incident Response and Root Cause Analysis
11When the AI recommends a containment action, what evidence did it exclude or downgrade in reaching that recommendation?
12Is this incident response suggestion based on the full context of my organisation's network design, or is it a generic playbook?
13Could this attack be using a novel technique that looks like a known attack family only in the first few steps?
14If I follow the AI's incident response steps, am I gathering evidence I will need for post-incident review, or only evidence that confirms the AI's initial hypothesis?
15Has the AI model been trained on incidents from my threat environment, or incidents from organisations with different threat actors?
16What would this incident look like if the AI had missed the real attack and misidentified the source?
17Is the AI recommending containment based on the blast radius it can measure, or is it blind to critical systems outside its visibility?
18When ChatGPT or Security Copilot suggests investigation steps, which of those steps am I performing because they are necessary, and which am I performing because the AI suggested them?
19Does the AI model understand the difference between a compromised account and a legitimate account that appears in unusual logs?
20If I skip the AI's recommended investigation step because I already know the answer, am I building a weaker evidence chain?
Vulnerability Assessment and Threat Modelling
21When AI scores a vulnerability as critical, is it critical in my environment or critical according to industry severity ratings that do not account for my asset's actual exposure?
22Has the AI made threat modelling assumptions about my organisation's architecture that are incorrect?
23Is the AI recommending a patch timeline based on known exploits, or based on the likelihood that this vulnerability will be exploited against systems like mine?
24What attack path did the AI assume when it recommended prioritising this vulnerability over others?
25Does the AI understand which of my critical assets have compensating controls that reduce the actual risk of this vulnerability?
26If an attacker has only the same information about my vulnerabilities as the AI model does, how much advantage would they have?
27Is the AI recommending a remediation sequence based on dependency chains in my environment, or a generic sequence?
28Could an attacker exploit this vulnerability in a way that the AI did not model because it is not documented in the public exploits the model was trained on?
29Does the vulnerability assessment account for my organisation's security controls, or only for the vulnerability in isolation?
30When the AI identifies a vulnerability class as high-risk, am I over-investing in that class because the AI is pattern-matching to a recent attack trend?
Security Architectsure and Decision-Making
31When AI recommends a security architecture change, is it recommending the best architecture for my threat model, or the most common architecture in its training data?
32Does this recommendation assume that I have the same security tools, team size, and detection coverage as the organisations the AI learned from?
33If I implement this AI recommendation, am I building security that defends against known attack patterns or security that stops thinking through new ones?
34Is the AI recommending a control that will create visibility I need for manual threat hunting, or is it recommending only what the AI itself needs to detect threats?
35What happens to my security posture if the AI model's training data becomes outdated or if an attacker learns my model's decision rules?
36Does the recommendation account for the business cost of implementation, or only for the risk reduction?
37If I automate this security decision based on the AI's recommendation, what manual review process do I still need?
38Is the AI recommending this architecture because it creates less alert volume for the security team, or because it is actually more secure?
39Does the architecture recommendation assume that my adversaries are the same as the adversaries in the AI's training data?
40When I implement this recommendation, will my team still have the skills to detect attacks that fall outside the AI's pattern?
How to use these questions
When alert fatigue is high, ask which signals you would investigate if the AI had not ranked them. Those are the signals your team is losing sight of.
Before you act on an AI incident response recommendation, trace it back to the raw evidence. If you cannot see the evidence clearly, you cannot verify the reasoning.
Schedule monthly threat hunts that are not AI-guided. These hunts will show you what your team's manual analysis finds that the AI misses.
Keep a log of AI recommendations that turned out to be wrong. Review them quarterly to understand where the AI is weakest in your environment.
Ask your threat intelligence team whether novel attack vectors in your threat landscape would be visible to your AI models. If not, design manual detection for them.