A Guide To Improving AI Productivity With Skeptical Intelligence

Artificial intelligence is quickly becoming the colleague we never asked for: answering our questions, drafting our documents, screening our résumés, and nudging our decisions. But while AI can be dazzlingly useful, it can also be dead wrong. The challenge is not simply whether to use AI, but how to think skeptically about the results it produces.

Skepticism is not a natural reflex. And when applied to AI, it requires a shift in focus: not just questioning the content of an answer, but the alignment between our problem and the machine’s interpretation of it.

Skepticism is both broader and narrower than critical thinking. It is both an approach to evaluating information and a mindset. Skepticism is not cynicism. Skeptics do not automatically believe that all information is false. Nor do they automatically believe that all information – especially when delivered by AI – is automatically accurate.

Here are five steps to improve your productivity with AI. Priyanka Shrivastava and I are working to validate this process with survey data and interviews. Please subscribe to this column for more.

1. Define the Problem

Ask two questions at the outset:

• What problem am I trying to solve?
• What problem is the AI actually solving?

These are rarely the same. If my problem is “How do I improve my leadership style?” but the AI’s problem is “Predict the next statistically likely sentence,” the gap matters. Even more worryingly is AI’s ambition to seduce the user into longer chats and repeat visits – into dependence. Skeptics begin by spotting the misalignment.

2. Check and Test Assumptions

Skeptics ask how the AI’s output was generated. Is the evidence it relies on relevant, accurate, and sufficient? If the model was trained on outdated data or biased samples, the evidence chain collapses.

AI delivers all answers with confidence, even if the underlying data is not certain. A skeptic selects the handful of most important sources for extra scrutiny. Did the key support for the argument come from a random opinion on social media or from an academic paper? (Depending on the problem you’re trying to solve, social media might be the ideal source.)

While most of AI’s most egregious hallucinations have been fixed, the root cause of these errors persists because of the way that AI works. It predicts the next word in a sequence based on probability. This makes its response rely heavily on the first few words of a response. For example, if there is a 50% chance of rain tomorrow and the question to AI is “Will it rain and why?”, the first words matter. “Yes, it will rain and here’s why” is as likely as “No, it will not rain and here’s why”

At the very least, a skeptic asks multiple sources to verify the accuracy of key assumptions. However, it is not enough to ask multiple AI engines the same question, because they all use similar datasets and prediction engines.

3. Spot Alternative Causes

Could there be another explanation for the AI’s result? AI can help find these. For example, flipping the question forces AI to reconsider data. “Why does a particular surgery have a 90% success rate?” can also be asked as “Why does a particular surgery have a 10% failure rate?”

AI shares many of the same flaws in its thinking as humans, along with a few additional ones. AI relies on historic averages. It will not deliver a creative, unlikely response. In fact, asking it to provide an innovative answer will cause the AI engine to look for popular responses with the word “creative” in them. For example, asking why a typical bridge might fail will give the most popular, likely answers. It will not provide improbable but highly damaging possibilities.

Moreover, AI does not understand causation; it only understands correlation. In other words, it knows that one idea often appears in the same sentence as another idea. AI does not understand why or how these two ideas are related. Understanding this relationship is the job of a “theory.” Again, AI can help by listing what theories might explain the relationship. This list points the skeptic towards a deeper, more accurate, more valid explanation.

4. Consider Counterarguments

What would a critic say? Test the AI’s result against external sources, domain experts, or rival models. If the answer can’t withstand opposition, it doesn’t deserve your decision. Again, AI can help by asking it to intentionally disprove its prior response.

A more proactive path is to phrase the question in opposing ways. For example, asking “Why is remote work improving employee productivity?” will generate a list of compelling arguments. Asking “Why is remote work undermining employee productivity?” will also generate a list of compelling arguments. This lists help skeptics weigh evidence.

5. Draw Your Own Conclusion

The machine proposes; you dispose. Accept, reject, or modify the output—but always be explicit about why. The act of articulating your assumption, logic, and judgment is the safeguard against passive adoption that could lead to the wrong conclusion.

The Payoff: A Repeatable Checklist

This five-step sequence is less about distrusting AI than about disciplining ourselves. AI will continue to expand in capability and scope, but its results will remain imperfect, partial, and—at times—profoundly misleading.

The way forward is not blind faith or blanket rejection, but a structured method for skeptical interrogation. Define the problem clearly. Question the assumptions. Demand sufficient evidence. And reserve the right to draw your own conclusion.

That is the essence of Skeptical Intelligence in the age of AI. This is the list that Priyanka and I will refine and illustrate over the coming months.