Using Local (AI) Compute To Reduce Reliance On Frontier Models via @sejournal, @chrisgreenseo
All Paid Media PPC News Social MediaAdvertising Video Advertising Columns Ask A PPC ExpertNEW PPC Pulse Guide AI Max Invalid Traffic Doubled, Weeks Before Black Friday Invalid traffic on AI Max search doubled to 5.28%. Get the channel-by-channel retail benchmarks before Black Friday. Digital Marketing All Digital Marketing Analytics & Data Ecommerce Lead Generation Content Marketing Social Platforms Google YouTube Reddit LinkedIn TikTokNEW WordPress Other CMS Platforms Guide Your ChatGPT Calls Are Hiding in the 'Direct/Other' Bucket Calls from ChatGPT, Claude, and Perplexity land in 'direct/other.' This checklist shows six steps to track them as their own source. SEJ Live Webinars Rundowns PodcastNEW Ebooks All Resources Webinar Defend Your 2027 Link Budget With Tactics That Still Work Join this webinar for a keep-or-cut list of off-page tactics and a step-by-step plan for running the ones that still work on your brand. Chris Green 43 seconds ago ⋅ 9 min read Chris Green Technical Director & Senior Consultant at Torque Partnership Bio Follow A lot of the current AI conversation assumes that “useful” AI means getting an agent to do the whole thing for you. There are some tasks where this does make a lot of sense – if done right – but there are many reasons why this isn’t the best option. If I want to extract & deduplicate a URL list collection from XML sitemaps, you don’t need a frontier model. What is better is to have an XML parser and deduplication script – for example – something cheap to (vibe)code, run, and ultimately is predictable. In SEO/GEO/AEO, there are tasks where a degree of interpretation is useful, but sending every request to a large remote model is not needed or even the best option. When I was experimenting with Exactly Matchy (that helps you understand if your content is retrievable by AI systems), I wanted something people could run without needing to grapple with APIs, credit cards, or general faffery. I knew that Chrome has a version of Gemini Nano (a tiny model, downloaded when needed), and wanted to leverage this to achieve simple tasks for you. The aim wasn’t to argue a small local model could replace a much larger model (it really can’t for a lot). It was to explore a more interesting question: How much useful work can we move closer to the user? Running AI locally is where we use our own hardware (phone, computer, laptop) to do the compute work without sending it off somewhere else to be processed. Using ChatGPT or Claude is simple – and often free to use – but it has drawbacks: Running most LLMs (models) involves a degree of complexity AND often a powerful machine capable of running the model you select. But what model do you select, and how do you know what the hardware is good for? These are tricky and important questions. Even after going through all this work, if you’re expecting a Claude-like experience, you’ll likely be frustrated because it still won’t measure up. This journey helped really show the difference between small and simple tasks. Exactly Matchy just selected passages from a page based on a really simple request and removes some friction for the user. I set out to see if it could complete small tasks to help someone understand technical SEO/GEO issues and whether they were an actual issue or not. Rather than just a technical checkbox that often leads to the wrong conclusion. Imagine a Chrome Extension which assists with Technical SEO, but more useful. There are some great Chrome Extensions out there that do relatively simple things, really well. But can AI assist in these small tasks to make you more effective? Would Nano be able to handle this? For example, comparing raw HTML with the rendered DOM produces a relatively small amount of evidence. With enough planning and deterministic processing, it is possible to then present that information to a model. Any experienced SEOs would find those pieces of data crucial to understanding whether that difference between raw/rendered is actually a problem or not. MOST SEO tools do a pretty bad job at helping people determine this without them doing the hard work! What if we gave this data to Gemini Nano, could we let it make that decision for us? Sadly not… This kind of decision-making is not an easy reasoning problem; it seems easy, but it isn’t simple. The model still needs to understand what this evidence means. It has to respect the facts (i.e., not conflict with them), combine multiple signals, and avoid inventing information or rationale that isn’t even present. Then it needs to make an accurate decision based on this. Gemini Nano is an intentionally small, fast model and then quantized so it fits in Chrome without slowing things down. It is intentionally the way it is, which isn’t ideal for what I was trying to achieve. In testing, Nano was useful at some tasks but unreliable at making the final judgment you could really trust. A stronger API (ChatGPT or Gemini) model handled the same evidence considerably better. When I gave it the deterministic details (i.e., those link attributes above), it did a really strong job at reasoning for you. Is this a failure of local AI? Maybe – I was a little disappointed, if not entirely surprised – but this was highly useful as a lesson in architecture for these kinds of problems. Throughout this process, the new extension has gradually settled into three layers: Fetching URLs, comparing HTML, checking HTTP responses, matching elements, identifying canonical relationships, and detecting whether a destination changed do not need probabilistic reasoning. If anything, asking an LLM to answer these questions is risky! Once the facts have already been established, Nano can turn a fairly ugly bundle of evidence into something a human can use quickly. If anything I’ve learned building teams, SEO/Serch programs, or training is that friction kills progress more than almost anything else. Presenting an easily readable passage rather than blocks of JSON or spreadsheets is highly valuable. I’d consider it a strength that Nano doesn’t have to make the decision – it’s actually easier in the long run. If there are technically complex, ambiguous details or we need some significant semantic or technical reasoning, a larger model does show its worth. We can provide the same structured evidence to Gemini, OpenAI, or another capable model. This is when you need to prioritize speed or supplement some knowledge gaps in a pretty reliable way. The important part is that the pipeline does not need to change. Only the model does, which impacts what exactly you get back. This all started as a test. I wanted to probe what Nano could do, so I benchmarked Nano’s reasoning ability against Gemini Flash and ChatGPT Luna. In each test, I pitched them against each other, treating the models equally in the test. This, I think, is where thinking about different AI models goes wrong. They do not need to replace frontier models to be useful! You certainly don’t need a frontier model for everything either! But how many people are going to know/understand this – and, to be honest, why should they? In the context I am testing here, the local model (Nano in this instance) needs to be good enough to take a meaningful amount of work off the user. There are several reasons this still makes local inference (via Nano) attractive:
Source: Search Engine Journal
This article has been carefully curated and reformatted for educational and informational purposes. Full credit goes to the original publisher.
📚 Visit more helpful articles on Joab Peters Blog
No comments