The AI Confidently Made Up Your Campaign Numbers

Your AI assistant just told you the campaign spent $150 last week. It has no idea what the campaign spent. It made the number up, and it sounded completely sure.

AI hallucinates campaign data when you ask it a question the data it retrieved cannot answer. You ask how an ad performed. The tool quietly hands back only the campaign's configuration: its name, its status, its daily budget. None of that is performance. So the model does what language models do best. It fills the gap with something that reads like an answer. "Your campaign spent $150 and drove 40 conversions." Clean. Specific. Entirely invented.

We have filed 60+ issues and fixed 30+ bugs building Campaign Agents across Google, Meta, LinkedIn, and Reddit. The hallucination bugs were a different kind of scary than the AI ad tools that fail silently I wrote about last time. A silent failure does the wrong thing quietly. A hallucination tells you the wrong thing loudly, and sounds more confident than the truth.

The Problem

You want to believe the intelligence lives in the model. You connect a capable LLM to your ad data, ask it a normal question, and it answers in fluent, confident prose. It feels like magic until you check the math.

The trap is that an LLM is a gap-filling machine. That is the whole job. Give it the first half of a sentence and it produces a likely second half. Give it a question about performance and only configuration data to work with, and it will produce a likely set of performance numbers. It is not lying. It has no concept of lying. It is completing a pattern, and "campaign report" is a pattern it has seen ten thousand times.

When you are building a tool people use to spend real money, "a likely set of numbers" is a disaster. The user does not see your tool call. They see a sentence. And the sentence is wrong in a way that looks exactly like right.

What Actually Happened

Three of these bugs taught me the shape of the problem.

The agent that invented spend (Issue #63). A user asked how their campaigns were doing. The agent called get_campaigns, which returns campaign metadata: name, status, daily budget. No spend. No impressions. No conversions. That tool was never meant to answer a performance question. But the user asked one, and the agent answered it anyway, fabricating spending and conversion figures out of the configuration it had. It also called the same tool twice in one turn, as if asking again would produce data that was never there to begin with.

The thin-data echo (Issue #64). Even when we routed performance questions to a real metrics tool, the agent gave generic advice. The reason was upstream of the model. Our Google Ads summarizer was compressing each campaign down to a list of boolean flags before the LLM ever saw it:

# What the model received. Thin, lossy, and already a conclusion.
summary["performance_flags"] = ["low_ctr", "lost_impression_share"]

Hand a model ["low_ctr"] and it cannot tell you anything specific, because nothing specific survived. So it echoes the cliché: "Your CTR is low, consider improving your ad copy." Technically not a hallucination. Practically useless, and indistinguishable from a tool that read nothing at all.

The wrong-year date (Issue #69). Relative dates are a quiet landmine. A user says "how did we do last month." Before that can touch a platform API, "last month" has to become an absolute range like 2026-05-01 to 2026-05-31. When that resolution picks the wrong year or a stale reference point, the agent queries a window with no data, gets little or nothing back, and then reports on it with total confidence. The format of the answer is perfect. The window underneath it is wrong.

Here is the pattern across all three:

What the user asked	What the data could answer	What the agent did
"How did my campaign perform?"	Only its config (name, budget, status)	Invented spend and conversions
"Why is this underperforming?"	Two boolean flags	Echoed generic advice
"How did we do last month?"	An empty, wrong-year window	Reported confidently on nothing

Every one of these is the same bug wearing a different coat. The model was asked to speak past the edge of what it knew, and it did, fluently.

Why This Is Harder Than It Looks

The instinct is to fix this with a sterner prompt. "Do not make up data." We added that line. It helps. It does not solve it, because you cannot prompt your way out of a missing-data problem. If the model is holding configuration and the user wants performance, "do not make things up" just turns a confident wrong answer into a confused one. The gap is still there.

Hallucination in an agent is almost never a model defect. It is a systems defect that shows up at the model. Trace any confident wrong answer back and you find one of three real causes:

• Wrong tool for the question. The data that came back cannot answer what was asked, and nothing stopped the agent from answering anyway.
• Lossy data. The right tool ran, but the pipeline crushed the result into something too thin to reason over before the model saw it.
• Wrong inputs. A date, an ID, or a filter resolved incorrectly, so the model got real-looking data about the wrong thing.

None of those are solved by swapping in a smarter model. A smarter model fed configuration data and asked for performance will produce a more convincing fabrication. That is the part that should keep you up at night. Raw capability makes a hallucination harder to catch, not easier.

How We Fixed It

We treated hallucination as a data and routing problem, not a prompting problem.

We made the correct path the path of least resistance. Performance questions route to a tool that returns real metrics, so the model is never standing in front of a question it has no data for. We enriched what that tool hands over, replacing the thin flags with full metrics, benchmark comparisons, and period-over-period deltas, so the model can say "your CTR is 2.8%, below the 3.2% search average, but up 15% from last period" instead of "your CTR is low." We hardened date resolution so relative ranges land in the right year before they ever reach a platform. And we drew a hard line in the guardrails: never infer spend, impressions, or conversions from metadata, and when results come back empty, widen the query rather than guess.

That is the approach. The implementation is one of more than 80 verification and grounding patterns we have built into Campaign Agents, because every platform and every question type has its own way of tempting the model past the edge of its data.

What This Means for AI Ad Management

If you are building in this space, the lesson is uncomfortable: the intelligence you are shipping lives in your plumbing, not in the model. The model is the easy 20%. The hard 80% is making sure that by the time a question reaches it, the only available answer is the true one. Route to the right tool. Hand over rich data, not conclusions. Resolve inputs correctly. Forbid inference where data should exist. And once the data is trustworthy, the more interesting question is what you should actually use AI for in an ad account, because grounding is the floor, not the ceiling.

If you are evaluating an AI ad tool, the question to ask is simple and revealing: "When the data does not support an answer, what does your agent say?" If the honest answer is "it answers anyway," keep looking. A tool you trust with budget should be able to say "I do not have that" out loud. Confidence is easy to generate. Being right when it would be easier to sound right is the entire job.