In the last year we've run AI-assisted analytics on customer data for businesses in leisure, restaurants, insurance, beauty, streaming and direct-to-consumer ecommerce.
The analysis was quick. A loyalty programme had first findings in front of its leadership team in 7 days. A research project produced 5 scored market studies in 48 hours.
But the same problems came up on almost every project. Teams used different definitions for the same metric; headline figures were wrong when we checked them against the source system; and reported results shrank once we compared them with a control group.
Industry surveys report the same problems. Supermetrics' 2026 Marketing Data Report found that only 6% of marketers have fully implemented AI. Adverity's 2025 research found CMOs estimate 45% of the data behind their marketing decisions is incomplete, inaccurate or out of date. 85% of the same CMOs say they trust it.
On our projects these problems showed up in the first week because that's when we tested the data.
Most teams had no agreed definition of their core metrics
Every project started with data the business already held. None of them needed a new platform. Most of them needed agreed definitions.
At a leisure business, sales and marketing quoted different contact rates from the same CRM. Each team used its own definition so they couldn't agree where sales were being lost. A restaurant loyalty programme had no shared definition of an active, lapsed or qualifying member. Both businesses agreed a single definition before we built any models.
AI produces findings quickly. Some of them are wrong in ways that look plausible.
With one client, leads from paid social appeared to convert better than leads from any other source. When we traced them back, many had been created on site and labelled as paid social in the CRM. We corrected the labels before the figure went into the media budget.
At a direct-to-consumer brand, spend per customer rose 14.4% in a year. All of the increase came from higher prices. Customers bought fewer items per order. Over the same period Google Analytics recorded purchases up 23% while actual orders fell 24%.
Each of these figures would have changed a budget decision if it had gone into a board pack unchecked. We now run 3 checks on any unusual figure before it goes to a client. We compare it with the source system, confirm the time period and check whether the way it was recorded changed during that period.
Gartner’s 2025 survey of CEOs and CFOs found that 54% are confident in their CMO’s ability to prove the value of marketing. Comparing customers who received a campaign with similar customers held back from it shows how much revenue the campaign caused. A finance director can plan a budget around that figure.
Automated CRM emails usually report revenue from every customer who received a message and later bought. Some of those customers would have bought anyway. A control group takes them out of the count.
When we measured automated emails this way, the results varied from flow to flow. Some produced a clear increase in revenue. Others couldn’t be read yet because too few customers had been held back to show an effect either way. Building a large enough control group can take months. The best time to set one up is when the email goes live.
Control groups also answer questions about the message itself. Testing a discounted message against a version without the discount shows whether the discount brings in more revenue than it costs in margin. Where both versions bring in the same revenue, the discount can come out. Timing can be tested in the same way, with each customer segment getting its own send point and its own control group.
A CRM team that reports against control groups can show the board which emails to keep, which to change and which to stop.
Most strategy projects run to a fixed date, with much of the time already committed to workshops and reviews. On our projects, AI-assisted research produced scored, sourced evidence on a market within 48 hours of kick-off. First findings on customer data reached leadership within 7 days. Gathering what’s already known took the first week. Workshops and expert panels could then spend their time on the questions that published research doesn’t cover.
Early findings also change where budget discussions start. When the value analysis arrives in the first week, spend can be ranked by how much each customer group is worth before the plan is written.
Where the customer data already sits in a cloud warehouse, the analysis can run inside it. That removes the usual extract-and-anonymise cycle, which can take about 2 weeks. Customer data stays inside the client’s own environment. The models stay there too, for the client’s team to keep using.
An experienced analyst directing AI agents can produce an evidence base in days that would normally need a team of analysts. That evidence is only useful if the client can trust each figure in it. We use 4 rules on every project.
Every figure must regenerate from a saved query. Any cut with fewer than 30 customers is flagged as directional. Each phase gets an independent check before anything reaches the client. Where no data exists, the report states the gap and leaves the figure blank.
Traceable evidence can also be checked by people outside the project. A client’s auditor or finance team can review the method against the queries behind it.
The same rules apply to published research. Where 2 credible sources disagree, both go into the evidence base with their sample sizes so the client can judge which one applies.
After 6 months of AI-assisted analytics, the projects that led to decisions had 3 things in common. The team agreed its definitions first, checked headline figures against the source system and measured results against a control group. Data is the moat.
Want to see how Data Agents can help you do more with less? Book a quick strategy call.