The gap between building and understanding
AutoML and AI-assisted feature engineering compress weeks into hours. That is genuinely useful. It is also how you end up owning a model you cannot fully explain, defending predictions you did not trace, and debugging failures you did not anticipate.
The critical instinct that catches these problems, the one that asks whether the training data actually represents the population, whether the evaluation metric measures the thing that matters, whether the model is confident for good reasons, grows from doing the slow work first. The tools have changed what slow means. The instinct still has to come from somewhere.
Where the problems appear
Edge cases are where the gap shows. A model that performs well on holdout data can fail systematically on a subgroup that was underrepresented in training, or optimize hard for a proxy metric that diverges from the actual goal under real-world conditions. These failures are not random. They follow from decisions made early, often by tooling that nobody interrogated.
The harder problem is that AI-generated model pipelines look plausible. The code runs. The metrics are reasonable. There is no obvious signal that something is wrong. Spotting the problem requires someone who knows what to question, not just what to measure.
What Steve works on with data science teams
Steve works with data science and AI teams on the critical thinking dimension of responsible AI practice. That means understanding what it takes to genuinely know a model's behavior, not just report on it. Monitoring tells you when something has gone wrong. Critical thinking tells you where to look and why.
The practical focus is on building teams that can challenge model behavior rather than just track it. That includes how to develop the habit of interrogating assumptions in a workflow that is designed for speed, and how to hold onto the slow instincts when the tools are pushing you to go faster.
Read the first chapter free
Steve's book, Cognitive Sovereignty, covers this in full. The first chapter takes about 20 minutes to read and is free.
Work with Steve
Steve speaks and consults with organizations working through exactly these challenges. See the Work with Me page for details.