The Question Changes the Retrieval

When an AI system answers a question, what it retrieves depends on what the question needs. Some questions call for explanation. Others call for specific things: products, projects, records, current data or outside confirmation. Treating every question as a search for the best explanatory page misses that, and it leads to the wrong diagnosis when a business doesn’t appear.

For a long time I asked the same thing whenever a company failed to show up in an AI answer to a question it should have owned. Do they have a good enough page on this?

It’s the natural question. It comes straight from how we’ve thought about search for twenty years: find the important question, write the best page answering it. Often it’s the right question. But the more citation data I looked at, the less it explained.

In reviewing which sources were cited across a large set of repeated AI queries, on several systems, the pattern that stood out wasn’t which domains won. It was that the kind of page being cited changed with the nature of the question. Some questions drew almost entirely on guides and expert commentary. Others pulled in individual real-world things. Some leaned on whatever was most current, and some produced a mixture of all three. The distinction became clearer once I stopped looking at domains and started looking at the kinds of pages being retrieved.

Three Questions, One Subject

Cybersecurity makes a good illustration, because the same company could reasonably want to appear for all three of these.

How should a mid-sized company approach zero-trust implementation?

This is a request for judgment. Useful sources are implementation frameworks, vendor-neutral guidance, accounts of what goes wrong and in what order. A strong explanatory page could satisfy most of it.

Which security platforms integrate with CrowdStrike and Okta?

The subject hasn’t changed, but the need has. Nobody asking this wants another essay on integration strategy. The useful evidence is specific: named products, integration pages, supported connectors, compatibility tables, version notes. To answer well, a system may need to find several concrete things and compare their attributes. A vendor could have the best thought leadership in the industry and still be poorly represented here if its integrations aren’t documented somewhere they can be found individually.

Which vulnerabilities are affecting VMware environments right now?

Now recency is central. The sources that matter are CVE records, vendor advisories, government vulnerability databases and recently updated analysis. A brilliant article from six months ago can be authoritative and still be the wrong evidence.

It’s the same field each time. One question asks for knowledge, one for objects and the relationships between them, and one for current state backed by an outside authority. “Cybersecurity authority” on its own tells you very little about what would need to be retrieved to answer any of them.

When the Question Is About Real Things

The object case is the one I find people underestimate, so it’s worth a second example.

Ask how to choose an architect for a coastal house and the useful material is advisory: process, experience with coastal sites, how materials hold up, how the working relationship should run. Ask instead to see contemporary coastal houses that use board-formed concrete, and the best evidence is no longer an article. It’s buildings. Individual projects, each with an architect, a location, a material specification, photographs and site conditions.

A practice could publish an excellent piece on concrete in marine environments and still not have the strongest evidence for that question, if its own project pages never say plainly what the house is made of, where it is and who designed it.

Push it one step further and ask which firms have recently won awards for hospitality projects. Answering that means connecting firms to projects to awards to dates, and the strongest corroboration may come from the awarding body or another independent source. That is object evidence and independent corroboration working together.

Seen this way, a website is more than a collection of pages. It’s a representation, good or poor, of a firm, its architects, projects, building types, locations, materials and awards, and different questions travel different routes through it. Whether the platform can express those relationships becomes a practical matter.

Expertise Has to Exist as Evidence Too

None of this is confined to businesses with products or inventory.

How should a manufacturer prepare for an ERP transformation? That’s methodology: planning, governance, change management, sequencing. Which consulting firms have implemented SAP S/4HANA for manufacturers? That needs proof the work was done, in the form of engagements, sectors, scope and outcomes. General SAP commentary doesn’t show it. Narrow the question again, to regulated medical manufacturing, and the evidence required is an intersection of technology, sector, regulatory setting, specific engagements and the people who led them.

As questions become more specific about experience, compatibility, examples or current state, the answer can depend increasingly on objects and their relationships, and less on general explanation alone. A firm can hold exactly that experience internally and expose almost none of it in a form anyone outside could retrieve.

The question determines what would constitute useful evidence.

That’s the idea I’d want a reader to take from the examples. It needs a qualification straight away, though. The question shapes what would be useful. It doesn’t decide mechanically what gets retrieved.

Some Questions Need Several Kinds at Once

The most interesting cases are the ones where a good answer draws on materially different kinds of evidence. I’ve been calling that heterogeneous retrieval, for want of a better term.

Take What does $2 million buy in this market? It reads as a simple question. Informationally it isn’t. Inside it are several others: what’s available around that price, what has sold recently, which neighborhoods appear at that level, how the properties differ, whether the market is moving, and what trade-offs a buyer should expect.

I’m not claiming any system runs those as literal searches. They’re dimensions of the original question. But decomposition is a real technique. Google documents what it calls query fan-out, where one question becomes a set of related searches, as part of how its generative Search features work.

The citation sets for questions like this show the result we would expect from a multi-part information problem: listings, sold records, market pages, explanatory articles and third-party reporting can appear side by side. That does not tell us which systems decomposed the question internally, or how. What it does suggest is that no single page held the strongest evidence for every dimension.

One thing I’ve noticed in those sets is a type of page that earns its place by bridging. A review of the year’s notable sales that links to each transaction. A market report tied to live inventory. An integration guide connected to the individual product documentation. A methodology page that points to the case studies behind it. I wouldn’t make a ranking claim about them. They’re structurally interesting because they give a retrieval system both the interpretation and a path to the concrete evidence underneath.

The System Matters as Well

Here is the second half of the qualification. The same question can produce noticeably different source mixes on different AI systems.

In the citation sets I’ve looked at, one system may lean on a handful of canonical explanatory pages where another, for the identical question, brings in individual objects, fresher results or a wider spread of outside sources. I’d be careful about reading personalities into that. These products change constantly, and what one does this quarter isn’t a fixed trait.

It’s also worth being exact about what the data shows. I can see which URLs are cited, what kinds of pages they are, how concentrated the set is, and how that differs between services. I can reasonably infer that different information needs favor different kinds of evidence. I can’t see hidden queries, ranking weights, or why one URL was chosen over another, and I don’t want to pretend otherwise.

What I’m comfortable saying is that there may be no single retrieval pattern attached to a question. The question shapes the evidence required. The system shapes how that evidence is found and assembled.

Not Every Gap Is a Content Gap

The practical consequence is about diagnosis.

When a business is missing from AI answers, the reflex is to call it a content gap and commission an article aimed at the question. Sometimes that’s right. If the question wanted explanation and there wasn’t a good one, write it.

But if what the question needed was a compatibility list, current availability, a record of completed work or independent confirmation, another explanatory article doesn’t address the problem. The evidence that fits the question still isn’t there. I’ve started thinking of this as retrieval fit: how closely the evidence a business has made available matches what a given question requires. Strong authority in a subject and poor fit for a particular question can sit together quite comfortably.

That isn’t an argument against writing. Explanation is still what a large share of questions call for. It’s an argument for working out what kind of problem you’re looking at before deciding what to build.

Start With the Information Need

I don’t think you need to know every hidden search or internal weight to act on this. Instead of asking only “what page should we create for this prompt?”, the more useful question is: what evidence would a system need to answer the question well, and have we made that evidence available?

I’ve found that to be a more productive way to think about AI retrieval than hunting for one content formula that works everywhere. Different questions create different information problems. I doubt one formula could answer all of them.