Google’s Illyes Unsure On Shifting unavailable_after Dates via @sejournal, @MattGSouthern
Webinar The AEO Playbook: Get Cited & Stay Visible Audit your AEO maturity and build a four-pillar citation strategy across AI Overviews, ChatGPT, Perplexity, and Claude. Guide 6 Ways To Prepare Your Business For AI In 2026 A practical checklist for using AI where it produces measurable pipeline and revenue impact in 2026. Rundown Why AI volume alone can't deliver personalization Your content architecture may be the real bottleneck. Get the Rundown on what needs to change in your content ops. Rundown Which AI Platform Wins in Your Clients' Verticals? ChatGPT, Perplexity, Claude, Gemini. Conversion leadership splits by industry. 20M data points, 20 GBP tactics. Webinar New AI Search & SEO KPIs: 4 Real Signals AI mentions and citations are benchmarks, not decisions. Get 4 traffic-predictive signals drawn from real bot data across hundreds of sites. 🔥SEJ Pro Course: Own Your Brand’s Promo Code & Coupon Search Results Before Parasites Do Google's Gary Illyes said he would need to check whether pushing an unavailable_after date forward on every renewal causes problems. He gave a first read. SEJ STAFF Matt G. Southern 47 seconds ago ⋅ 3 min read SEJ STAFF Matt G. Southern Senior News Writer at Search Engine Journal Bio Follow Google’s Gary Illyes said on LinkedIn that he’d have to check whether a site can safely extend an unavailable_after date. His first read, which he called a gut feeling, was that pushing the date forward is fine, but he left open whether the tag can be trusted for pages that expire and then get renewed. SEO consultant Javier Lorente Murillo reached out to Google’s Search team with a question about a classifieds site that has a steady core of around 5,000 indexed URLs and adds approximately 10,000 new listings each month. Most of these listings stay live for 24 to 72 hours. He planned to use unavailable_after on short-lived pages to avoid 404 errors and manage crawl budget. Because users can renew ads, his system would dynamically update the expiry date. He asked whether Googlebot would start to ignore a date that keeps changing with each renewal. “that’s a great question… I have no idea, I have to check.” “my gut feeling is that it’s fine to push forward the unavailable_after date BUT you need to keep in mind that we’ll need to crawl the page again to “see” the new date. it doesn’t have implications on anything but index selection, where it acts as a “you can drop this now” signal.” In a later response to another comment on the same post, he said Google would fetch the page instead of checking it cheaply: “we rarely see HEAD requests so we’d still download the content.” Google’s robots meta tag documentation explains ‘unavailable_after’ as a rule instructing Google not to display a page in search results after a specified date and time. The same page says Googlebot will crawl that URL considerably less often after that date. The page also talks about the recrawl point Illyes mentioned, highlighting that Robots meta tags and X-Robots-Tag headers are detected when a URL is crawled. This means that a changed date will only appear on the next fetch. That part of his answer echoes what Google already states. The claim that the rule touches nothing beyond index selection is his own, and it doesn’t appear in the docs. Illyes didn’t say what happens if the last date Google saw passes before it gets back to the page. That is the case he said he would have to check. With the renewal case unanswered, there’s nothing from Google to plan around. What decides the outcome is how often Google crawls the page, and a site owner can’t control that. As of publication, Google’s documentation doesn’t cover an expiry date that changes on every renewal. The first call is what you’re asking the unavailable_after tag to do. Letting a page drop out of search results after a set date is what Google documents it for. Keeping a page alive while leaning on the tag to manage crawl is the case that’s still open. Read Full Bio SEJ STAFF Matt G. Southern Senior News Writer at Search Engine Journal See short video versions of news stories on YouTube and TikTok. Matt G. Southern is the Senior News Writer at ... Learn how to connect search, AI, and PPC into one unstoppable strategy. Lighthouse Fails Your Llms.txt Without Markdown Links Why Data Integrity Is The New Technical SEO: From Crawling To Trust Why Your Pages Are Stuck In Crawled-Currently Not Indexed & What To Do About It Join 75,000+ Digital Leaders. Learn how to connect search, AI, and PPC into one unstoppable strategy. Learn how to connect search, AI, and PPC into one unstoppable strategy. In a world ruled by algorithms, SEJ brings timely, relevant information for SEOs, marketers, and entrepreneurs to optimize and grow their businesses -- and careers.
Source: Search Engine Journal
This article has been carefully curated and reformatted for educational and informational purposes. Full credit goes to the original publisher.
📚 Visit more helpful articles on Joab Peters Blog
No comments