Your AI Vendor's Breach Is Your Incident

Production AI delivery now runs on a supply chain, and the past week showed how far a breach inside it travels. Notion disclosed that a security incident at Fireworks AI — an inference vendor serving models inside its product — may have exposed unattributable Notion AI chat transcripts and touched fewer than 1% of its workspaces S1. Fireworks' own bulletin attributes the attack to a financially motivated group selling unauthorized LLM access into illicit routing gateways, and traces the entry path through a breached third-party cloud provider into its development machines on October 24 S2.

Read together, the two disclosures are a field study in what actually limits blast radius when an AI vendor is compromised.

The stored-data path sets the blast radius

Fireworks reports no detected impact to its zero-data-retention products — Inference, Training SDK and Nexus S2. The damage ran through stored data: the bulletin counts account and billing information among the affected categories, and for a limited set of accounts, secrets, datasets and incomplete model weights; the affected internal application also exposed resource metadata such as resource identifiers, display names and creation times. Payment card numbers were not stored and were not impacted S2.

Notion's exposure came from stored transcripts feeding its Insights classification feature. The notice reports two facts side by side: the affected set was under 1% of workspaces, and the exposed transcripts were "unattributable" — no user or workspace labels, no tool call results, nothing beyond the chat text itself tying a conversation to a company S1. The notice never says the labeling gap caused the small number; don't import that causation. What is documented is the data's content — and the residual risk is that free text. Notion tells anyone who pasted passwords or API keys into Notion AI to rotate them immediately S1.

The takeaway for teams shipping AI features is blunt: classify every vendor data path as zero-retention or stored, and give stored paths your database threat model, not a hand-wave.

The exit runbook is a deliverable

Notion's FAQ dates the disconnect to November 3: after learning of potential incidents earlier in the month, it cut all traffic to Fireworks, deleted its data on the vendor side, and halted all training runs S1. Fireworks then notified Notion on November 6 that malicious actors may have accessed potential customer data, and confirmed the new dataset on November 8 S1. Notion has disabled Insights and stopped serving open-weight models through Fireworks.

That sequence only works with a rehearsed kill switch: a per-feature map of which vendor touches which data, and deletion rights you have actually exercised. Fireworks' own remediation — revoked and replaced credentials, network isolation, and disabling self-serve dataset uploads and downloads for Batch Inference and dataset-based training S2 — means customers on those paths need fallbacks too.

Zero-trust has a verifier problem

The architectural debate is catching up with practice. Writing in IEEE Spectrum, Vishal Sharma of Queen's University Belfast argues that zero-trust's "never trust, always verify" strains under agentic systems: the policy enforcement point itself must be trusted, and QUB experiments measured orchestrated de-authentication attacks executing in as little as two to three seconds — little time to react S3. Recovery compounds it: an agent gone rogue is "not a containable small task" and its impact needs sweeping across every touchpoint S3.

The Fireworks chain — customer, vendor, vendor's cloud provider, developer machines — is exactly that multi-hop shape. Trust did not degrade gracefully along it, and it will not along yours. Ship the exit runbook with the feature.