If a Human Can't Do It With Your Data, AI Can't Either

A few months ago I was working with a client in the Utilities sector. The task sounded simple: use company data to create test cases for a system that was to be supported by generative AI. The data was there, the systems were there, the budget was there. One thing was missing — something nobody had thought to check.

Nobody verified whether a human could actually do it.

We Took It on Faith That the Data Was Good

This is how most AI projects I see from the inside actually look. The organization has data in its systems. Someone in a meeting says: "we have data." Someone else adds: "we just need to plug in AI." And the project kicks off.

In this case, the data was supposed to be used to generate test cases. We took it on faith that the data was correct and sufficient. Nobody questioned that assumption — why would they? After all, the data had been in the system for years; people used it every day.

The problem only surfaced when we asked a domain expert — someone with years of experience in that organization — to manually create a few test cases using the same data.

He couldn't do it.

The Data Existed, but Not for This

It wasn't that the data was bad in a technical sense. It was in the system, it had some structure, someone had entered it at some point. But it wasn't suitable for the task we wanted to perform. Context was missing, links between records were absent, change history was nonexistent. The expert looked at the data and saw fragments — not a complete picture from which he could build a test scenario.

And the organization expected generative AI to do it "automagically."

The Test That Should Come First

The conclusion I reached after this project is simple and brutally practical: before you spend budget on AI, give the same task to a human. Not one person — ideally two or three from different departments. Give them the same data, the same objective, and observe.

If a domain expert cannot complete the task using that data, then the problem isn't with the AI model. The problem lies in the data, in the assumptions, or in the task itself.

This is not a technical test. It is an organizational test. It checks whether the company understands its own data well enough to use it — whether by a human or a machine.

Why Companies Skip This Step

Because it's uncomfortable. Because it means that before we start "deploying AI," we first have to do the boring, tedious work: review the data, understand its quality, fill in the gaps, create documentation. That's not sexy. You can't put it on a slide for the board with the words "AI transformation."

But without this step, everything else is building on sand.

I see this pattern repeatedly: companies invest in a proof of concept that works on clean, prepared demo data. Then they try to move it to real company data and everything falls apart. Not because AI is weak. Because real data is nothing like the demo.

What We're Really Testing When We Test AI

Every AI project is fundamentally a test of the organization. It tests whether the company:

• Knows what data it has and where that data lives.
• Understands the quality of that data — not declaratively, but practically.
• Can formulate a task precisely enough that a human could execute it.
• Has experts who can verify the output.

If the answer to any of these is "no" — it's not the time to deploy AI. It's the time for a process and data audit that shows you where you really stand.

The Industry Says: "Data Quality Is Key"

And it's right — but it stops halfway. Reports from Deloitte, McKinsey, and Gartner talk about data quality as a foundation. But nobody says it plainly: check whether a human can work with that data before you spend the AI budget.

That's the missing step. It's not just about whether the data is "clean" in a technical sense — no duplicates, formats match. It's about whether the data actually allows the task you want to delegate to a machine. Because if an expert with 15 years of experience can't create a test case from it, then no model — not GPT-4, not Claude, not whatever comes out next year — will do it either.

Practical Advice

Before every AI project, do one simple exercise. Take the task you want to assign to AI. Give it to the best expert in the company. Give them exactly the same data you planned to feed the model. And observe.

If the expert completes the task smoothly — you have a shot at a meaningful AI deployment. If the expert struggles, asks for additional data, searches for context outside the system — you have a problem that needs solving before you even think about AI.

This is not an AI test. It is a test of your organization's readiness.

If you want to check how ready your company is to deploy AI on real data and processes, I invite you to a conversation — Leszek Giza.