Q3 AI Visibility: AI Citations, Brand Mentions & Content Refreshes That Work via @sejournal, @AirOpsHQ
AI Visibility Rankings Aren’t Stable – New Research Shows It’s Mostly Statistical Noise Reddit Tops Search Results Across All Niches After May Core Update The 2026 State of AI Search: How Modern Brands Stay Visible A Smarter SEO Content Audit: Aligning For Performance, Purpose & LLM Visibility AI SEO: Writing That’s Specific May Get Cited More Q3 AI Visibility: AI Citations, Brand Mentions & Content Refreshes That Work Reading Now AI Visibility Rankings Aren’t Stable – New Research Shows It’s Mostly Statistical Noise Reddit Tops Search Results Across All Niches After May Core Update The 2026 State of AI Search: How Modern Brands Stay Visible A Smarter SEO Content Audit: Aligning For Performance, Purpose & LLM Visibility AI SEO: Writing That’s Specific May Get Cited More AI Visibility Rankings Aren’t Stable – New Research Shows It’s Mostly Statistical Noise AI visibility numbers move between runs, so a single reading can mislead. A new paper offers a stopping rule for when rankings are finally trustworthy. Matt G. Southern 1.7K Reads AI visibility tracking data isn’t entirely reliable. Because generative models often produce different responses, the citation shares and rankings on your dashboard are merely snapshots of a continuously changing target, not fixed facts. A difference between you and a competitor could be genuine or just fluctuation between measurements. A new IQRush paper released on April 11, provides a method to distinguish these, showing that no fixed amount of data can definitively settle the question. The paper is by Ron Sielinski, who co-founded IQRush, who sell software that measures AI visibility the way the paper argues you should. The reason it’s worth your time is that a separate team published a similar repeated-measurement finding in April, so IQRush is not the only one making this case. Repeatedly querying SearchGPT, Gemini, or Perplexity with the same question can produce different sources each time. They’re built to add some randomness to each response, so each citation is just one of many possible URLs it could have pulled. A prior paper by the same author explored this variability, showing that, for example, when testing SearchGPT on running gear, Tom’s Guide made up about 9.5% of citations, while Runner’s World accounted for roughly 6.0%. On the dashboard, Tom’s Guide appeared more often, but the large margin of error meant the figures overlapped. With only one sample, it wasn’t accurate to say Tom’s Guide outperformed Runner’s World, as the 3.5-point difference was within the margin of error. The new paper aims to prevent this mistake by addressing a simple yet often overlooked question: How much data is needed before rankings are truly meaningful? The answer has two parts, and both need to be true for a ranking to be reliable. First, the order must stop changing. In the beginning, rankings may change frequently as new answers are added because no site has a clear edge yet. It’s only after enough answers are collected that the top sites start to stand out clearly, allowing the order to stabilize. Also, it’s important that the top sites are well apart; if they’re very close, the ranking might not be meaningful, as a tight competition doesn’t really show who’s truly ahead. The paper looks at whether the difference between the top sites is bigger than the margin of error for each. When it is, the ranking reflects a real difference. When it isn’t, it’s probably just statistical noise. Both conditions need to be true at the same time, neither alone is enough. In 30 platform-topic tests, the number of answers needed for both conditions to be met ranged from 33 to 94, counting only answers with citations. Three out of 30 didn’t reach this point even after 125 questions, all on SearchGPT, where top sites were too similar to tell apart. There is no single cutoff applicable everywhere; what works for one platform and topic may not suit another. In January, I discussed SparkToro’s discovery that AI tools give a different list of recommended brands more than 99% of the time you ask the same question. That article left one question unanswered: how many times do you need to ask before the results stabilize? This paper offers the clearest answer I’ve come across. Rand Fishkin, who led that study, shares some helpful advice. Before spending any money on tracking AI visibility, he suggests making sure your provider “shows their math.” The IQRush paper is a great way to do this because it provides a simple stopping rule, so you don’t have to rely solely on intuition about how many runs are enough. It also fits a run of studies SEJ has covered over the past year, each reporting AI citation numbers as if they were fixed. This one turns around, examines the measurement itself, and asks whether those numbers are stable enough to compare in the first place. The number on your dashboard is just a single sample. Before trusting it, check whether your tracker performs the same check repeatedly and reports a range, or if it pulls data once and shows a clean figure. The clean figure can actually be a warning sign, not reassurance. A gain after a content change is easy to misinterpret. For example, a three-point increase in your SearchGPT citation share might seem like proof that your effort paid off, but such a change can fall within the natural variability of successive runs, according to the original paper’s data. To claim the win, measure before and after more than once each. A single before-and-after reading cannot separate your change from ordinary noise. The platform you are measuring changes how much data you need, and not in the way you would guess. It comes down to how much independent information each answer carries, not how many citations it hands you. Gemini piles citations onto the same handful of sites within a single answer, so many of those citations tell you the same thing. SearchGPT gives fewer citations per answer but spreads them out, so each answer carries more independent information than the raw count suggests. The same number of answers on two engines does not buy the same confidence, and a budget that settles Gemini can leave you guessing on SearchGPT. Sometimes the honest answer is that you cannot say yet. Three of the 30 tests never cleanly separated their top sites within the budget. For those, the right call is to hold, not to publish a ranking the data cannot support. A tracker that can tell you “not enough data” is worth more than one that prints a confident order every time you ask. The top of the ranking is the part you can most defend. With enough answers, the leaders pull away from the middle and tail, though even they are not exact. The margins of error widen fast below the front, until neighboring positions are a coin flip, and even the top 10 were not spotless, with the typical margin of error on a top-10 site running about five positions and one in five wider than 10. Trust the leaders, treat the middle and bottom as rough, and do not report exact positions past the front of the list. None of this comes from a finished, peer-reviewed study. It is a preprint built on 30 platform-topic tests across three engines, using questions generated by ChatGPT rather than real user searches, over a single stretch of collection. The exact numbers will not transfer cleanly to your topics, so treat them as the shape of the problem, not a lookup table. Those counts include only answers that carried citations, which matters most on SearchGPT, because a share of its questions return no citations at all. In one topic, 125 questions produced 104 usable answers, a 17% miss, so you would need to submit more questions than those totals suggest. The check on the method is internal, too. The paper compares a ranking it calls early against that same collection’s final ranking, not against any outside ground truth. That tests whether the stopping rule is consistent with itself, which is why the matching result from the unaffiliated team does real work here. The authors of that April paper, Julius Schulte, Malte Bleeker, and Philipp Kaufmann, are researchers at the University of St. Gallen. They ran a separate dataset and reached the same verdict, that a single reading is unreliable and you have to sample an engine repeatedly to trust what it tells you. The paper stops short of the thing most people will want, which is a way to know your run budget before you start collecting. Sielinski leaves that for later work and notes that the number depends on the shape of each platform’s citation pattern, so a single universal budget probably is not coming. The bigger change is that AI visibility reporting is headed the way ad and analytics reporting already went, toward numbers that carry a margin of error instead of a false decimal point. That is happening while the basic plumbing is still missing, since Search Console still won’t tell you which clicks came from AI. Until it does, the job falls on you to run the check more than once and report the range, not the single number your dashboard hands you. SE Ranking found Reddit gained top 3 share across all 20 niches after Google's May core update, with smaller moves in YMYL categories. Matt G. Southern 2.2K Reads An SE Ranking analysis of 100,000 keywords found Reddit grew its top 3 presence in all 20 niches tracked after the May core update. For transparency, SE Ranking sells rank-tracking and AI visibility tools, and the data comes from its own keyword-monitoring platform. Data suggests Reddit grew most in niches where people look for personal experience. Here’s a closer look at the data broken down by category. Reddit’s overall top 3 share rose to 10.24% after May, up from 8.56% after March and 9.19% after December. That’s about one in every ten top 3 spots in their data. Reddit also took the #1 spot more often. It held the top result for 13,872 keywords after May, up 54% from 8,993 after March. That contrasts with what happened after the March core update, where Amsive found Reddit and similar UGC sites lost US search visibility while brand sites gained. SE Ranking’s March data showed Reddit’s top 3 share declining from its December level, but moved back the other way in May data. YouTube’s top 3 organic share dropped to 2.14% after May, down from 2.50% after March and 2.40% after December. Data indicates that YouTube results may be appearing more often in video SERP features and less often in regular organic positions. The analysis covers organic blue links only, so any YouTube presence in video carousels or other features isn’t counted. Top 3 monopolies, where one domain holds all three top spots for a keyword, dropped to 1.99% of keywords after May, down from 3.24% after March. YouTube is still the domain most likely to hold a monopoly, but its share of those keywords dropped from 15.5% after March to 15.4% after May. For overall volatility, SE Ranking found May landed between March and December. After May, 76.03% of top 3 URLs changed position and 88.39% of top 10 URLs changed. Both figures were lower than March but higher than December. About one in five top 10 pages (19.87%) disappeared from the top 100. That’s lower than the 24.10% that dropped out after March but higher than December’s 14.70%.
Source: Search Engine Journal
This article has been carefully curated and reformatted for educational and informational purposes. Full credit goes to the original publisher.
📚 Visit more helpful articles on Joab Peters Blog
No comments