Your keyword research is optimised for a behaviour that no longer exists.
The tools are precise, the methodology is rigorous but the outputs are completely disconnected from how people actually discover information now. Search Engine Land recently documented something most SEO teams suspect but can't prove: users prompt AI systems in fundamentally different patterns than they type into search bars. Healthcare queries are symptom-based; software queries are comparison-driven; financial queries are scenario-specific. None of these patterns map cleanly to keyword clusters.
Traditional keyword research assumes a user typing fragmented phrases into a search box. "Best CRM software." "CRM pricing comparison." "Salesforce alternatives." The tools are excellent at surfacing these patterns. They can tell you search volume, competition, seasonal variation.
What they can't tell you is how someone prompts Claude or Gemini or ChatGPT when they actually want to make a decision.
The prompts are longer, they're conversational and they include context the user would never type into Google. "I run a 50-person sales team and we're outgrowing HubSpot. What should I look at that integrates with our existing Slack workflows and doesn't require a dedicated admin?" That prompt contains zero traditional keywords but it contains enormous commercial intent.
Google's own Search Central team has published guidance acknowledging this shift. They're telling website owners to optimise for generative AI features. The subtext is clear: the discovery mechanism is changing and the old signals matter less.
The teams seeing results in AI search aren't running more sophisticated keyword analysis. They're mining different sources entirely.
Call centre transcripts. CRM notes. Customer service chat logs. Support tickets. Sales call recordings.
These sources contain the actual language customers use when they're trying to solve a problem. Not the compressed, decontextualised fragments they type into search. The full thought, the specific situation and the constraints they're working within.
One e-commerce brand we work with found that their highest-converting customer segment consistently described their problem using language that appeared nowhere in their keyword research. The phrases were too long, too specific, too conversational to register as "keywords." But they were exactly how those customers prompted AI systems for recommendations.
The insight wasn't expensive to find. The data already existed in their support tickets. Nobody had thought to mine it for search optimisation because the keyword tools couldn't process it.
The Search Engine Land analysis revealed something valuable: prompt patterns aren't random. They cluster by industry in predictable ways.
Healthcare prompts lead with symptoms, not conditions. Users don't prompt "diabetes management." They prompt "I'm waking up multiple times at night to use the bathroom and I'm always thirsty."
Software prompts are comparison-driven with constraints. Not "best project management tool" but "something lighter than Monday.com that my team will actually use and that works with our existing Google Workspace setup."
Financial prompts are scenario-based. Not "retirement calculator" but "I'm 45 with £200K saved and I want to retire at 60. What am I missing?"
Each of these patterns requires different content architecture. The healthcare brand needs symptom-first content structure. The software company needs comparison frameworks that surface constraint-matching features. The financial services firm needs scenario-based calculators with narrative explanations.
This emerges only from understanding how your specific customers articulate their problems when they're not constrained by a search box.
This isn't a mandate to rebuild your entire search strategy. The shift is more surgical.
Start with your customer-facing data. Pull the last 500 support tickets. The last 200 sales calls. The last quarter of live chat transcripts. These already exist. Nobody is analysing them for prompt patterns.
Look for the long-form articulations. The sentences where customers explain their situation in full. The questions they ask before they ask for a solution. The constraints they mention unprompted.
Cluster these by intent, not by keyword. What problem are they trying to solve? What situation are they in? What outcome are they hoping for?
Then map your existing content against these clusters. You'll find gaps: content that ranks well for keywords but doesn't address the actual prompts; pages optimised for fragments that miss the conversational queries entirely.
The rebuild means reframing what you already have around the actual questions people ask.
You can't measure AI search visibility the way you measure traditional search rankings. The metrics don't exist yet. Google's Search Console is beginning to surface AI-related data, but the reporting lags the behaviour change significantly.
This creates a legitimate business case problem. How do you justify shifting methodology when you can't prove the new approach works with the same precision as the old one?
The answer is inference:
David Dobrin made this point recently on The Precision Brief: competitive advantage comes from the quality of context you feed into those models. Everyone has the same models. Your proprietary customer language data is the differentiator. For AI search visibility, that context is your proprietary customer language data. Your competitors can buy the same keyword tools but they can't buy your support tickets.
Right now, most teams are still optimising for keyword patterns. The ones mining customer language data for prompt patterns have a structural advantage.
That advantage grows. The earlier you understand how your customers actually articulate their problems, the earlier your content starts appearing in AI-generated responses. The earlier that happens, the more signal you accumulate about what's working.
The teams that wait for better measurement tools before making the shift will find themselves optimising for the new behaviour after their competitors have already occupied the space.
The keyword research process your team runs is measuring something that matters less every quarter. The data you need to replace it already exists in your CRM, your support system, your call recordings. It's waiting to be mined for patterns your keyword tools will never surface.
Want to see how Data Agents can help you do more with less? Book a quick strategy call.