AI Delivery Now Ships Two Artifacts: Goodput and Guidance

Production AI delivery is settling on two artifacts. The first is an open field guide to inference engineering — numbers stamped as of October 2026 — arguing that platform teams become inference teams S1. The second surfaced on Hacker News as a Chrome-and-Edge Show HN, crossing 200 points within four hours S4: a preview release installing curated web-platform guidance into coding agents S3. Together they show what production AI delivery now ships: a serving stack measured in goodput, a context pack measured in tokens.

Platform teams are becoming inference teams

The guide's job description is unglamorous: an inference engineer picks a model, gets it onto the right GPU, keeps it serving, makes it faster and cheaper, and gets paged when it breaks. That is SRE for a new workload; most skills carry over, with sharp translations. Autoscaling on CPU becomes scaling on queue depth and KV cache use, with cold starts in minutes. Rollouts become a version matrix of model, hardware and engine build. Latency percentiles become time to first token, time per output token and goodput S1.

The new math is memory. Weights are a fixed footprint — an 8B model at two bytes per parameter needs about 16 GB — while the KV cache grows with context length times concurrent conversations. The guide's example: Llama 3 70B at FP16 costs roughly 320 KB of cache per token, so one 128K-token conversation demands about 40 GB S1.

The labor market confirms the discipline is young: an August 2026 hand count of public careers boards found roughly 58 open roles in the entire visible market, 24 at OpenAI and Anthropic, no two companies using the same title — and posted bands reaching $850,000 S2.

The second artifact is a guidance pack

Modern Web Guidance starts from a plain diagnosis: coding agents default to older patterns because training data contains vast amounts of legacy code. Its answer: 212 modern web features and 180 real-world developer use cases, installable like a dependency, pruned by evals that drop content models already know S3.

The Show HN post adds receipts: agents run against 200 tasks with and without the skill — about 1,700 assertions — from 55 percent pass unguided to 92 percent guided S4. That matters more than the feature list: it treats an agent's context as a maintained artifact — versioned, eval-filtered, pinned like a package. For forward-deployed work, that is the shape worth copying: customer-specific guidance — prompts, runbooks, playbooks — should live as a reviewed dependency, diffs read and updates gated, not as drifting wiki prose.

What actually transfers

The common thread is translation, not reinvention. Three takeaways:

Inventory the delta before hiring a new title. Scheduling, rollouts, observability and FinOps carry into inference; the scarce additions are GPU memory math, cache-aware routing and honest benchmarking — and with no standard title, search several job names S2.

Treat agent guidance as a reviewed dependency. A vendor pack shapes what your agents ship, so install it pinned, read diffs, review updates like code.

Report in the new units, failures included: the guide asks engineers to describe one failure with real numbers and to explain cost per million tokens to a finance partner S1. Data Gravity's census separately found the market tiny and the pay enormous; its author adds a gloss — that inflated vendor numbers are "one more reason the people who can measure honestly are worth what they cost" S2. Treat that as his thesis, not a measured wage equation. Either way, the practical point stands: goodput, not marketing, is the unit of delivery now.