What We Can Learn From Evolving ChatGPT Fan-Out Queries via @sejournal, @lilyraynyc
All Paid Media PPC News Social MediaAdvertising Video Advertising Columns Ask A PPC ExpertNEW PPC Pulse Rundown Why AI volume alone can't deliver personalization Your content architecture may be the real bottleneck. Get the Rundown on what needs to change in your content ops. Digital Marketing All Digital Marketing Analytics & Data Ecommerce Lead Generation Content Marketing Social Platforms Google YouTube Reddit LinkedIn TikTokNEW WordPress Other CMS Platforms Webinar New AI Search & SEO KPIs: 4 Real Signals AI mentions and citations are benchmarks, not decisions. Get 4 traffic-predictive signals drawn from real bot data across hundreds of sites. SEJ Live Webinars Rundowns PodcastNEW Ebooks All Resources SEJ Live AI Search Visibility for Local Business August 26 | How ChatGPT, Gemini, and AI Overviews choose a single local recommendation - plus reviews, GBP, and attribution. ChatGPT is scoping its fan-out searches to sites it already trusts, and I think the site: operator is a method of reducing spammy results. See what the recent industry research reveals. VIP CONTRIBUTOR Lily Ray 8 seconds ago ⋅ 20 min read VIP CONTRIBUTOR Lily Ray Founder of Algorythmic at Algorythmic Bio Follow Over the past few months, I’ve been paying close attention to how ChatGPT’s fan-out queries have been evolving, as some interesting developments have taken place that I believe are changing the quality of ChatGPT’s responses. I’ve been working on this article for a while, but this one has been particularly difficult to write, because as with all things in AI search, the information changes more quickly than I can finish writing about it. That’s definitely been true for how OpenAI appears to be tweaking and refining its process of retrieving information via web search (RAG), and especially for how heavily ChatGPT has started relying on site: searches in fan-out queries, potentially using them to curate results from higher-quality sources. The TL;DR: I think OpenAI is using fan-out queries, and the site: operator in particular, as one method of reducing spammy outputs in their answers derived from internet content. I believe it’s their effort to improve the quality of retrieved sources while taking early steps to combat spam and low-quality information in their results. It reminds me of what Google has tested with E-E-A-T, but ChatGPT style. I think analyzing fan-out queries matters because ChatGPT is using search engines to retrieve the results it uses to formulate an answer, which means this is fundamentally an SEO problem. Every word the model chooses to put into a fan-out query, and every search operator it uses, tells us something about what the model is looking for and where in the search results it expects to find it. When it scopes a search with site:, adds the word “official,” or points at a specific subreddit, the model is telling us what kind of content it believes will best answer the user’s question. Those queries are the closest thing we have to understanding why ChatGPT pulls in the information that it does, and I think there is a lot we can learn from unpacking them. There are a few folks in the industry who have done great work sharing their findings on the inner workings of ChatGPT: reading the raw traffic, scraping the conversation files, and pulling fan-outs out of the API. Their datasets have started to converge on the same findings, and this article combines the learnings from their research with my own observations from watching fan-out behavior over time, using a combination of Peec AI, Profound, the Resoneo plugin, FanoutFox, and Google Search Console. So I aim to do two things with this article: First, lay out what everyone has actually found, in one place, with the numbers attributed to whoever ran them. Then give you my read on what’s evolving and why I think it’s important. Not all searches on ChatGPT use web search. Anything contained in its training data can be answered quickly without using RAG, and OpenAI’s free or cheaper models are more likely to rely on training data to answer questions quickly, as it costs them less money to generate. When the question requires up-to-date knowledge, ChatGPT will use web search (retrieval-augmented generation or RAG) to pull information from a variety of sources, including external search engine data and its own internal index (Labrador). When you do ask ChatGPT a question that triggers a web search, it starts by deconstructing your prompt into a set of its own background searches (fan-out queries), runs them in parallel, and synthesizes an answer from whatever comes back. Monitoring how fan-out queries change over time tells us a lot about how OpenAI is tweaking the model’s search behavior to try to produce better results. I think watching this space gives us a big clue about what they were hoping to achieve with each model update. It’s important to start by defining two words commonly thrown around in our space, without it always being understood what the nuance is between them. Retrieved means a page ChatGPT fetched while running its fan-out queries. Cited means a page that made it into the visible answer as a link. Citation and retrieval can behave differently, and right now they appear to be moving in opposite directions: the number of “retrieved” URLs in ChatGPT’s responses is growing, while independent measurements show the number of unique domains cited per response falling over the same period. While more pages are being considered for the answer, fewer pages get cited and credited. While reading any study about AI search, it’s worth asking whether the article refers to retrieved URLs or cited URLs, and if it’s discussing citations, for which prompts those citations are appearing. In many cases, the discrepancies I’ve seen between fan-out studies come down to one measuring retrieval and the other measuring citations. To read the actual fan-out queries (not just the final answer), I’ve been using a combination of a few tools: Peec AI and Profound both offer fan-out queries for tracked prompts, and the free Resoneo ChatGPT Chrome plugin and FanoutFox (shown below) both surface the queries the model runs and the sources it pulls. These plugins make it easier to watch the model “think” through its searches in real time. Timing matters here too: it’s essential to consider how and when ChatGPT releases new models, and which models and tiers are most commonly used by the majority of its users. ChatGPT 5.6 ships in more than one variant: Sol is the standard version, and the cheaper “Luna” variant is what rolled out around the start of August as the new default model for Free and Go users. That distinction is important when you read the studies below, because they aren’t all measuring the same model or tier. And according to Olivier de Segonzac’s breakdown in Search Engine Land, more than 90% of ChatGPT’s weekly users are on the free plan. So whatever the free default does when it searches, it’s now most likely what the large majority of ChatGPT users get. A few patterns stood out to me early on, before I went looking at anyone else’s data: A screenshot from the Resoneo plugin showing how the prompt “tell me about legal benefits for disabled veterans” generated ChatGPT fan-out queries all limited to specific .gov sites (Image Credit: Lily Ray) The net effect, at least in what I looked at, appears to be more ChatGPT visibility for high-authority sites and trusted brands, and fewer citations for everyone else. Below, I’ll highlight a few recent studies on this topic and what they found. Several awesome folks in our space have been measuring this independently, with different tools and different collection methods. David Konitzny at Peec AI ran the numbers the day ChatGPT 5.6 became the default: the share of prompts with only a single fan-out query dropped from 94.0% to 43.5%, average retrieved sources roughly doubled from about 12 to 24, prompts needing a second fan-out iteration went from about 5% to 33.5%, and the site: operator went from appearing in roughly 0.3% of fan-outs to about 23%. Basically, ChatGPT is becoming more precise and robust in its searching process. Chris Long at Nectiv, comparing roughly 4,000 prompts on 5.6 Sol against his own 2025 baseline, found average fan-out queries per prompt went from 2.17 to 7.61, the longest query chain went from 4 searches to 29, and “site:,” “official,” and “gov” all landed as top unigrams, with site: in 64% of queries. There’s a big gap between the site: search figures in David’s and Chris’s articles, and it could be explained by the fan-out query collection method: Chris’s consultancy pulls fan-outs from OpenAI’s API, while several of the other tools in this space extract them from the ChatGPT consumer interface. While the API is clean and repeatable, the UI method may be closer to what real users actually get (with certain limitations like personalization, which no tool can track effectively). It’s worth checking which method a study used to understand why different studies may show discrepancies. In both studies, however, the share of site: searches in query fan-outs increased substantially. David also found product pages now make up 16.39% of retrieved pages, moving ahead of listicles, which tracks with what I’d been seeing anecdotally: ChatGPT 5.6 appears to go straight to brands and manufacturers for specs and pricing rather than routing everything through roundup articles and other listicles, which can be self-serving and prone to manipulation. The page types losing share of retrievals (listicles, how-to guides, and comparison pages in particular) also happen to be the formats most heavily spammed for GEO over the past year or two. In my talks throughout this year, I’ve shared how these exact page types cause SEO and AI search problems. Olivier de Segonzac and the Resoneo team, who have done some of the most detailed reverse-engineering of the retrieval architecture out there, found that the unique domains cited per response dropped from 19 to 15 after the 5.3 update, confirming the pattern from the other direction: retrieval counts grew while citation counts dropped. Ahrefs’ most-cited-domains data shows where the surviving citations land most frequently: Reddit, Wikipedia, Forbes, Merriam-Webster, Consumer Reports, Healthline, and Walmart. Suganthan Mohanadasan has been coming at it from the network-traffic side, and his findings are useful to understand: ChatGPT decides before it searches. Across 57 conversations and 3,554 retrieved pages, 21 of 27 initial queries contained brand names the user never mentioned in their prompt, across 11 of 13 product categories. For example, when searching “the best AI note-taking app,” the first fan-out query already contained the words “Granola,” “Notion AI,” “Otter,” “Fireflies,” “Fathom,” “Mem,” and “Limitless.” Below is a snippet from Suganthan’s recent article, shared with permission: Being the brand mentioned in that first query is the real goal: brands named in the fan-out were cited 68.9% of the time, while pages that were only fetched were cited 2.1% of the time, and only 110 of the 3,554 retrieved pages, about 3.1%, made it into an answer at all. Suganthan also explained ChatGPT’s routing logic within fan-out queries: facts route to official pages, and opinions route to reviews and Reddit. So ChatGPT will look to different types of sites, and distinct pages within them, depending on which information helps to best answer the user’s question. As with all studies, it’s helpful to read his methodology alongside those numbers, which he states himself: a single account, based in Dubai, sampled in July 2026, weighted toward software and AI tools, with the brand-injection pattern measured across 27 initial queries. That’s a relatively small sample size, but it’s worth citing because the direction also matches what the larger datasets show, and what I see in my own testing. Although the above researchers used different tools, different models, and different collection methods, their findings still line up across four common patterns: The skew is toward surfacing higher-quality, recognizable sites, and that trend is continuing with newer ChatGPT models. I find all of this interesting because it aligns closely with one of the components of Google’s search ranking systems I’ve spent the most time studying: E-E-A-T (experience, expertise, authoritativeness, and trustworthiness). I think OpenAI may be using fan-out queries as a method of ensuring users get high-quality, trustworthy, authoritative information, while suppressing spammy and highly manipulated articles (such as self-promotional listicles and other types of self-serving content). I think the site: operator is functioning (at least in part) as a spam filter. For a while, ChatGPT would frequently retrieve and cite the exact kind of spammy, self-serving, manipulative content that we’ve already become quite familiar with in the SEO world. Evaluating the quality of a random open-web page in real time is genuinely hard and expensive, and at ChatGPT’s volume you’d have to do it hundreds of millions of times a day. Narrowing fan-out queries to .gov domains, established review platforms, big recognizable brands, and official sources is presumably a much cheaper way to get most of the same outcome. Instead of judging whether an unknown page is trustworthy, the model appears to be sidestepping the problem entirely by only looking for information in places it already trusts. Google spent years building systems to answer “is this source authoritative enough for this query?” and what OpenAI appears to be doing is a distilled version of this same process, relying heavily on the site: operator and other keywords that influence which sources get retrieved. For example: pure facts go to the brand’s own domain. Health and legal questions go to .gov sites. Opinions go to Reddit and established review platforms. While Google uses complex ranking algorithms to surface high-quality, trustworthy, and authoritative pages, ChatGPT can refine its fan-out queries to ensure the search results pull from those trusted sources. Adding “.gov” to certain queries (YMYL, perhaps?) is what I find intriguing, because it reminds me of how Google elevates .gov (and other high-authority) sites in its results for certain queries or during times of crisis. For example, during the Covid era, I shared a lot of research showing how Google elevated the official FDA, CDC, and other high-authority and government sites in the search results for health-related queries: Research from my 2022 MozCon presentation (Image Credit: Lily Ray) The .gov-only pattern in that batch of legal prompts (shown below) feels like a big change on ChatGPT’s part, as it separates the most “popular” or “optimized” pages on the internet from the official authority sites. On almost any consumer legal topic, the popular, top-ranking pages shown in search come from law firm blogs, not government agencies. Combined with the page types losing retrieval share, that strikes me as a way for ChatGPT to cut through the noise and elevate official information instead.
Source: Search Engine Journal
This article has been carefully curated and reformatted for educational and informational purposes. Full credit goes to the original publisher.
📚 Visit more helpful articles on Joab Peters Blog
No comments