I’m setting up AI agents for marketing analytics for several clients right now, and we’re starting to get a practical feel for what actually works when assigning tasks to them.

We’ve already found a few useful principles. But one idea from Ilya Krasinsky made the whole thing sharper.

Here’s the point.

You can brief an agent like this: “Do X.”

A bit better: “Do X and give me contradictions, open questions, and quick wins.”

But the version that works much better is: "Do X. Then try to disprove your own result.”

Why does this matter?

The problem with LLMs is not that they’re stupid. The problem is that they’re very good at making the first plausible answer look coherent.

If your prompt doesn’t include doubt, checks, alternatives, or failure modes, the model often has very little reason to attack its own hypothesis.

So instead of testing the idea, it polishes it until it looks convincing.

Which is great if you want a nice-looking answer.

Less great if you want something that survives contact with reality.

That’s why it helps to give the agent not only an execution task, but also a falsification task.

This is basically one of the core ideas of scientific thinking: a good hypothesis is not strong because it explains everything beautifully. It’s strong because it can be challenged.

Falsifiability doesn’t weaken the idea. It makes it more useful.

What you can ask the agent to do:

  • list its assumptions
  • identify what it didn’t consider
  • name 3 reasons why the solution might fail
  • check alternative explanations
  • show where it produced the appearance of a result, not the result itself

Bad version:

“Analyze retention and suggest how to improve it.”

Better version:

“Analyze retention. Then challenge your own conclusions: find 5 reasons why this hypothesis might be false, check alternative explanations, and separately list what you did not verify.”

A good agent is not just an obedient executor.

It’s leverage.

But without critique, it will happily walk you down the path of coherence - and not necessarily toward the result.

Want all my best GA4-BQ queries in one place? I turned them into a Chrome extension — top SQL queries you can search and copy in seconds.

Go here to install it for FREE.

Prefer the web version? It’s here.