ChatGPT’s Search Index Serves Small Sites Too, Data Shows via @sejournal, @MattGSouthern
All Paid Media PPC News Social MediaAdvertising Video Advertising Columns Ask A PPC ExpertNEW PPC Pulse Webinar Google Local Services Ads Are Moving To PMax: What To Check First Prepare for Google's LSA move into Performance Max with a before-and-after audit checklist from CallRail. Digital Marketing All Digital Marketing Analytics & Data Ecommerce Lead Generation Content Marketing Social Platforms Google YouTube Reddit LinkedIn TikTokNEW WordPress Other CMS Platforms Webinar New AI Search & SEO KPIs: 4 Real Signals AI mentions and citations are benchmarks, not decisions. Get 4 traffic-predictive signals drawn from real bot data across hundreds of sites. SEJ Live Webinars Rundowns PodcastNEW Ebooks All Resources Webinar AI Search Visibility for Local Business August 26 | How ChatGPT, Gemini, and AI Overviews choose a single local recommendation - plus reviews, GBP, and attribution. Advertising On SEJ Case Study: B2B SaaS Banner Ads Webinar Google Local Services Ads Are Moving To PMax: What To Check First Prepare for Google's LSA move into Performance Max with a before-and-after audit checklist from CallRail. 🔥[Live 8/12 with Loren Baker] Ecommerce SEO: Own your "brand +promo code" search. New data finds no difference in how OpenAI's in-house search index served licensed and unlicensed sites, and shows where ChatGPT's snippets come from. SEJ STAFF Matt G. Southern 3 seconds ago ⋅ 5 min read SEJ STAFF Matt G. Southern Senior News Writer at Search Engine Journal Bio Follow Resoneo says hundreds of outlets with no OpenAI content deal were served by OpenAI’s in-house search index exactly the way its licensed partners were. In its free-account data, that index handled most ChatGPT search results. The French SEO consultancy read 1,249 ChatGPT answers captured in July. Resoneo sells SEO consulting and gives away the Chrome extension that captured the data. The finding backs a correction Suganthan Mohanadasan published in July, after he initially read the index as largely closed to smaller sites. ChatGPT’s server stream tagged each web result with the name of the pipeline that fetched it, and one of the four values was ‘labrador,’ OpenAI’s own index. When comparing pages from that pipeline, Resoneo found that a licensing deal didn’t change how a page was served. It was the same format, same length, and same freshness for both partners and non-partners. Resoneo describes labrador as an index topped up with press feeds and open science archives. They mention that OpenAI can access it directly without needing to pay a third party, which is what sets it apart. In Resoneo’s free-account data, questions with settled answers, local businesses, and products appeared through that index almost every time. The news results were split fairly evenly between the index and what was scraped from Google. For paid accounts in thinking mode, Google scraping provided around 75% of the 16,407 search results that Resoneo recorded, while the in-house index made up about 24%. Source: Resoneo. Pipeline classifications are based on its reverse-engineering of ChatGPT network traffic. Mohanadasan described the same index as an allowlist of established publishers in June, after examining ChatGPT’s network traffic. He mentioned that it “looks like a licensed tier,” including domains like Reuters, The Guardian, the WSJ, and Wikipedia. On July 14, he took that back. A reader from Italy, using a free account, sent him captures showing that every publisher citation went through the same pipeline, including small Italian sites. Mohanadasan re-ran his tests, acknowledged in his summary table that he “over-reached” with the tier claim, and mentioned that the licensing deals are genuine, but the tier reading was based on viewing just one account’s perspective as representative of the entire situation. The two conducted various tests, each with a different size. Resoneo’s dataset includes both free and paid accounts, multiple countries, and logged-out sessions, with the same prompts replayed across different account types. Mohanadasan’s counts came from one account and he calls them directional, though his correction also draws on captures from two other readers’ accounts. Resoneo credits Mohanadasan’s work as the foundation for their own efforts. Around July 21, according to Resoneo, OpenAI stopped tagging each search result with the name of the system that fetched it, which is the tag both investigations had been reading. Resoneo reviewed 534 pages that ChatGPT cited, and compared each one with the snippets stored in OpenAI’s index. Out of the 463 pages with an H1 heading, 387 snippets included it, or 83.6%. The snippet gets cut off just after 200 characters, usually from the beginning of the page content rather than the meta description, which the Google-scrape pipeline still captures roughly one out of three times. The median H1 was 51 characters long, which leaves roughly 150 characters of page content. A section kicker appears before the H1 on 29% of pages and takes up 18 characters. A publication date appears on 11% of pages, using 25 characters, and the alt text of the first image appears on 9% of pages and can take 50 characters on its own. In the sample, one out of every seven pages didn’t have any H1 markup. Resoneo mentions that in those cases, the snippet begins with whatever subheading the template provides. Sites without an OpenAI content deal still appear in the index that manages most free-account ChatGPT results. Resoneo’s findings support this, as does Mohanadasan’s own update. After his retest, he recommended checking with multiple accounts to get a clearer picture, since a single account only reveals how ChatGPT interacted with that one account. The index stores a title and about 200 characters from the page. Anything a template prints above the first paragraph uses up part of that. Resoneo didn’t test whether changing it makes a page more likely to get cited. Publishers sign content deals with OpenAI for several reasons. Appearing in ChatGPT’s answers to free users looks like a weak one, because sites without a deal were already in the index that handles most of those answers. Whether a deal helps a page get cited more often is a different question, and neither investigation looked into it. Resoneo focused on how pages were stored and served. As of publication, OpenAI’s crawler page doesn’t detail its in-house index or specify what its publisher agreements include. Resoneo notes that partner articles reach OpenAI through a feed rather than a crawl, so a deal could change how content gets there. Read Full Bio SEJ STAFF Matt G. Southern Senior News Writer at Search Engine Journal See short video versions of news stories on YouTube and TikTok. Matt G. Southern is the Senior News Writer at ... Learn how to connect search, AI, and PPC into one unstoppable strategy. Local SEO: Which AI Platforms Drive the Most Leads by Industry The Science Of What AI Actually Rewards The Verified Source Pack Agents Trust First Join 75,000+ Digital Leaders. Learn how to connect search, AI, and PPC into one unstoppable strategy. Learn how to connect search, AI, and PPC into one unstoppable strategy. In a world ruled by algorithms, SEJ brings timely, relevant information for SEOs, marketers, and entrepreneurs to optimize and grow their businesses -- and careers.
Source: Search Engine Journal
This article has been carefully curated and reformatted for educational and informational purposes. Full credit goes to the original publisher.
📚 Visit more helpful articles on Joab Peters Blog
No comments