Aug 1, 2026 / AI Engineering The context length printed on a model card is a training decision, not a hard ceiling. Most open weight models trained at 32K can be run at 64K or 128K by rescaling the rotary position embeddings at inference time, and this is exactly what happens when you set a rope scaling factor or hand a model to YaRN. The mechanism is real and it works, which is precisely why it is dangerous. There is no assertion, no truncation warning, no crash. You set the factor, the model accepts a longer prompt, and it generates fluent text. What changes is subtler: the model's ability to locate and reason over specific tokens degrades as the effective positions get stretched further from what it saw in training. On a needle in a haystack test the retrieval accuracy sags in the middle of the extended window. On real work the answers get plausibly wrong, citing the right document but the wrong number, summarizing the first half of a contract and forgetting the clause on page forty. This post is about that trade specifically on Apple Silicon, where unified memory makes a 128K context physically possible on a single machine and therefore makes the temptation to just crank the factor very real. We cover what RoPE scaling actually does to positions, why YaRN degrades more gracefully than naive linear scaling, how to measure the degradation instead of hoping it is not there, and where the honest ceiling sits for a model you did not train yourself.
Jul 31, 2026 / Headless Commerce
The speed argument for headless commerce rests on one thing: the storefront is static enough that a CDN can serve it from the edge, close to the user, without a round trip to an origin. Then the business asks for personalization, a logged in greeting, cart aware pricing, a member only price, a segment specific banner, and that single requirement threatens to collapse the entire model, because a page that differs per user is a page the edge cannot share across users. The naive reaction is to mark the page uncacheable and send every request to origin, which throws away the reason you went headless in the first place. The correct reaction is to stop treating the page as one cacheable unit and start separating what is shared by everyone from the small fragments that actually differ per user. That separation is the whole discipline of caching a personalized storefront, and it comes down to three mechanisms that most teams half implement and then wonder why their hit rate is poor or their users see each other's data. The cache key decides what counts as the same page. The Vary header decides which request dimensions split the cache. And hole punching decides how a mostly shared page carries a per user fragment without becoming per user itself. Get these three right and a personalized storefront still serves the bulk of its bytes from the edge. Get them wrong and you either leak one customer's cart to another or cache nothing at all.
Jul 31, 2026 / AI Models
Most coding leaderboards report one number, the percent of problems solved, and that number hides the failure that actually costs you in an agent. Aider Polyglot is built to expose it. The benchmark draws 225 of the hardest Exercism exercises across C++, Go, Java, JavaScript, Python, and Rust, gives the model two attempts with the failing unit test output fed back after the first, and crucially it requires the model to return its change in a structured edit format the harness has to apply mechanically. That second requirement is the whole point. A model can reason its way to a correct fix and still score zero because it emitted a malformed diff the tool could not apply, and that is not a benchmark artifact, it is exactly what happens inside a real coding agent when a model gets the logic right and the patch format wrong. We ran GPT-5.6 Sol and Grok 4.5 across the full polyglot set and separated the two signals the leaderboard blends together: raw solve rate, and edit format compliance. The headline solved numbers are close. The gap that decides which one to wire into an agent shows up when you look at how often each model produced an edit that applied cleanly on the first try, and which languages dragged each one down.
Jul 31, 2026 / AI Engineering In memory prefix caching is the well known win for local inference: hold a stable system prompt in the KV cache and never prefill it twice. The part nobody plans for is what happens when the process exits. The cache lives in RAM, so it dies with the server, and the very next request after a restart, a crash, or a fresh deploy pays the full prefill cost again as if the cache never existed. On a long context that is not a rounding error. A 60K token document or repository context can cost several seconds of prefill on an M5 Max, and if your workload restarts often, ships on a schedule, or runs as a short lived batch job, you are paying that tax over and over for a prefix that never actually changed. MLX has a direct answer that most local setups leave switched off: the prompt cache is serializable. You can precompute the KV cache for a fixed prefix once, write it to a safetensors file on disk, and load it back on startup so the first token comes out at warm cache speed even though the process is brand new. This post is about that mechanism specifically, the disk persistence layer rather than the in memory hit. We look at how the cache file is produced and consumed, what the load actually costs versus a cold prefill, the constraints that make a saved cache silently wrong if you ignore them, and the workloads where persisting to disk is the difference between a viable local service and one that stutters on every boot.
Jul 30, 2026 / AI Models
SWE-bench Verified has a scoreboard problem: the top models all cluster above seventy percent, which means the benchmark can no longer tell you which one to actually deploy. SWE-bench Pro exists to fix that. It is built from long horizon tasks that can take a professional engineer hours to days, it requires patches that span multiple files rather than a single localized fix, and it was designed from the start against data contamination by including held out and commercial repositories the models could not have trained on. The result is a benchmark where the frontier scores in the low twenties rather than the low seventies, and where the spread between models is wide enough to be a real signal again. We ran Claude Opus 5 and Gemini 3.5 Pro on the public set and looked past the headline solved rate at the thing that actually predicts production behavior: how each model fails on a task that touches six files, where getting the change right in one place means getting it right in five others too. This is the benchmark that separates a model that can patch a function from a model that can carry a change across a codebase, and that distinction is exactly what agentic coding lives or dies on.
Jul 30, 2026 / Headless Commerce
Selling internationally on a packaged storefront is mostly a settings screen: turn on the currencies, let the platform convert, and the theme shows the right number. Go headless and the convenience disappears, because now your own frontend is responsible for asking the API for a price in the customer's currency, displaying it with the correct rounding, carrying that currency all the way through the cart and into checkout, and reconciling what the customer paid against what the store settles in. Each of those steps has a way to go wrong that a themed store never exposed you to, and the most common one is subtle: the storefront requests a presentment currency and the API quietly returns the amount in the store's base currency anyway, so the customer sees a euro sign in front of a dollar number. Add duties and import taxes on a cross border order and the surface expands again, because now the total the customer sees at checkout has to include or exclude a duty depending on whether you sell delivered duty paid or delivered duty unpaid, and getting that wrong means either a surprised customer or a margin you did not plan to give away. This post is about the architecture that keeps a headless international store honest: which system owns the converted price, how presentment currency actually flows through the Storefront API, why rounding is a real rule and not a rounding error, and how duties change the checkout total.
Jul 30, 2026 / AI Engineering The mental model most people bring to a vision language model is wrong in a way that costs them latency. They picture an image as a single input, roughly one unit of work, the way a text token is one unit of work. What actually happens is that the vision encoder chops the image into patches, and a high resolution photo can turn into hundreds or thousands of visual tokens that the language model must then prefill through before it generates anything at all. On a cloud endpoint you never feel this, because someone else eats the prefill and bills you a flat per image rate. Run the same model on your own M5 Max and the cost becomes visible immediately: the model loads fine, it holds in unified memory with room to spare, and then a single screenshot at native resolution takes several seconds to first token because you just asked the machine to prefill four thousand tokens it manufactured out of one picture. This post is about that gap. We look at where the memory actually goes when you load a VLM locally, why image tokens and not model weights are usually your latency problem, how the resolution setting is the one knob that moves both accuracy and speed, and when a local VLM is the right call versus a cloud vision API. The economics of local vision are real, but only if you understand that the image, not the prompt, is the expensive part.
Jul 29, 2026 / AI Models
Coding leaderboards have converged on one kind of task: fix a bug in a real software repository, usually a web application, usually in Python. That is a real skill and it is not the only one. SciCode measures a different thing entirely, whether a model can read a scientific problem from physics, chemistry, biology, or materials science and produce numerical code that actually computes the right answer, often across a chain of dependent subproblems where getting step three right requires step two to be correct first. It is a harder axis than repository bug fixing because the model has to hold a domain concept, translate it into the correct algorithm, and implement that algorithm without a test suite handing it the shape of the solution. We ran GPT-5.6 Sol against Gemini 3.5 Pro on SciCode and measured not just the headline solved rate but the collapse point, the subproblem depth at which each model starts propagating an early error through the rest of the chain. The gap that matters is not who tops the aggregate; it is which model degrades more gracefully when a multi step derivation goes wrong, because that is the failure mode you inherit when you point either one at real scientific tooling.
Jul 29, 2026 / AI Engineering The naive way to serve a fine tune is to merge the adapter into the base weights and load the merged model, which is fine until you have twenty fine tunes and a machine that can only hold two merged copies of a 70B model in memory at once. LoRA exists precisely so you do not have to do that. The adapter is a few hundred megabytes of low rank matrices that sit on top of a frozen base, which means one resident base model can back many adapters if your serving layer knows how to keep the base loaded and swap or batch the adapters around it. The gap between knowing that and running it in production is where most teams stall, because the obvious implementation reloads the whole base every time a request wants a different adapter, and that throws away the entire advantage. This post is about serving many LoRA fine tunes from a single M5 Max: how much memory each resident adapter actually costs, how fast you can bring a cold adapter in from disk, why merged serving falls over at scale, and the batched adapter pattern that lets requests for different fine tunes share the same forward pass. The economics only work if the base stays put and the adapters move cheaply, so the whole design is about protecting that invariant.
Jul 29, 2026 / Headless Commerce
The decision to leave Salesforce Commerce Cloud is usually made on cost and velocity, and by the time it reaches engineering it has hardened into a deadline. That is where replatforming projects go wrong, because moving from SFCC to a headless Shopify Plus stack is not a migration in the copy the data and flip the switch sense; it is a rebuild of the parts of your commerce logic that lived inside SFCC cartridges and pipelines, wrapped around a data migration that is the easy part by comparison. SFCC gave you a monolith where the storefront, the business logic, and the platform were fused, and headless Shopify Plus deliberately unfuses them: Shopify becomes the commerce engine behind an API, and the storefront becomes your own application. Everything that made SFCC feel complete, the cartridge ecosystem, the pipeline customizations, the server side rendering baked in, becomes something you now own explicitly. This playbook walks the migration in the order that actually de-risks it: what maps cleanly from the SFCC data model to Shopify, what has to be rebuilt rather than ported, how to protect the SEO equity that a careless cutover destroys, and the sequencing that lets you move without a big bang launch. The projects that fail treat this as a data problem. The ones that succeed treat it as a rebuild with a data migration attached.
Jul 28, 2026 / AI Models
Almost every agentic coding benchmark you have seen a model quote is Python, because SWE-bench is Python and it became the number everyone optimizes. That leaves a blind spot the size of most production code, because the repository your agent actually has to fix is as likely to be Java, Go, Rust, or TypeScript as it is Python, and a model that tops the Python leaderboard can quietly fall apart in a language it saw less of in training. Multi-SWE-bench exists to measure exactly that: it takes the same real bug, real repository, real test suite format that made SWE-bench credible and spreads it across seven languages, so the score reflects whether a model can resolve an issue in a codebase that is not written in the language it is best at. We ran Claude Sonnet 5, a mid tier frontier model, against GPT-5.6 Sol on the multilingual splits and measured not just the headline resolved rate but where each model drops off by language, because the gap between a model's Python score and its Rust score is the number that predicts how your agent behaves on the repo you actually have rather than the one the benchmark authors happened to pick.
Jul 28, 2026 / AI Engineering A single M5 Max is a genuinely good inference box for a steady stream of requests, and if your load were flat you would never need anything else. Real load is not flat. It has a baseline you can size hardware against and spikes you cannot, and the spike is where a local only setup fails: the queue depth climbs, time to first token blows past your latency budget, and users wait behind a machine that is already at full decode. The fix is not a bigger Mac, because you would be buying capacity for a peak that shows up a few hours a week and sitting idle the rest of the time. The fix is a hybrid: keep the baseline on local hardware where each token is nearly free and no data leaves your network, and burst only the overflow to a cloud API that you pay for by the token and only when you actually need it. The design problem is the router. A router that spills too early throws away the cost advantage you built the local tier for, and one that spills too late lets the queue hurt users before it reacts. This post covers how to set the spill threshold against queue depth rather than raw request rate, how to keep the cloud path from leaking data you meant to keep in house, what the blended cost actually looks like, and when a hybrid is worth its extra failure modes versus when one tier is the honest answer.