Data privacy and compliance

Training and retention are different questions

"Do you train on my data?" and "do you store my data?" are independent questions, and a vendor can answer no to the first while answering yes to the second.

Training means your inputs and outputs enter a dataset used to update model weights. Once that has happened it is effectively irreversible — you cannot delete a gradient. Retention means your inputs and outputs are written to storage the provider controls, typically for abuse monitoring, debugging, or because a stateful API feature requires it. Retention is bounded, deletable, and enumerable in a DPA.

For the major API providers the default is: no training, bounded retention. OpenAI does not use API data to train its models unless you opt in, and holds abuse-monitoring logs up to 30 days unless law requires longer; some features store data longer by design. Anthropic's commercial products do not train on inputs or outputs by default, but thumbs up/down feedback is a separate category — it captures the whole conversation, is de-linked from user and customer IDs, may be used for training, and is retained up to five years. Admins can disable it org-wide. Google's paid Gemini tier excludes prompts and responses from product improvement; the unpaid tier does not, and lets human reviewers read and annotate them. Exceptions cut across the tier line: Grounding with Google Search stores data 30 days on both.

The consumer/enterprise split

Consumer chat products and developer APIs from the same vendor sit under different terms. Consumer tiers frequently do use conversations for model improvement — sometimes opt-in, sometimes opt-out, sometimes changed by policy update with a deadline to choose. Free API tiers can be worse than consumer tiers, because "free" is often priced by the data.

Zero data retention

Under zero data retention (ZDR), the provider persists no request or response content — inference happens in memory, nothing is written to disk. It is not a checkbox: it requires approval, usually per-organization.

• Stateful features stop working. Anything that stores content server-side for later retrieval is unavailable or degraded. Under OpenAI's ZDR, store is forced to false on Chat Completions and Responses whatever the request asks for, and the Assistants API, Conversations, ChatKit threads, Vector Stores, video, and fine-tuning jobs are ineligible outright. The general rule: anything that keeps state "until deleted" rather than for a fixed window cannot be zero-retention.
• Support gets harder. The provider cannot see the request, so escalations run on your reconstruction of what you think you sent.
• Abuse monitoring shifts to you. Providers granting ZDR generally expect the customer to take on misuse detection, sometimes as a contract term.

ZDR is right for regulated content — PHI, legal privilege, source code under strict IP controls — and over-engineering for a support-ticket classifier. Decide per workload, not per company.

Data residency and regional inference

Two different guarantees hide behind "our data stays in the EU":

• Storage residency: logs, stored files, and account metadata live in a named region.
• Regional inference (processing residency): the forward pass itself runs on hardware in that region.

Providers ship these separately with different region lists: storage residency across many countries, regional processing for a much smaller set and only for supported models. Ask which you are being sold, for which models, and what happens when you call a model with no regional deployment — silent fallback to another region is the failure mode to rule out. Several providers also require abuse-monitoring modifications as a precondition for non-default regions, so residency and ZDR are one negotiation.

Subprocessor chains

Gateways, inference marketplaces, and agent frameworks with hosted tools add hops your DPA may not name: your app → gateway (routing, caching, spend limits) → model provider or inference host → possibly a cloud region run by a fourth party. Each hop is a potential controller/processor relationship, a potential log, and a potential jurisdiction.

• Gateway prompt logging. Many gateways log prompts and completions by default, because that is how their analytics and evaluation features work. It is usually a per-project setting, and the default is often "on."
• Semantic caching stores prompt content — and serves one tenant's output to another's request if the cache is not partitioned correctly. Confirm cache scope.
• Marketplace inference hosts aggregate independent providers behind one API. The binding terms are the downstream provider's, and differ per model. Where a host publishes per-provider retention and training flags and lets you restrict routing, use that filter rather than its headline policy.
• Open-weights ≠ private. Running a permissively licensed model on someone else's GPU gives you the licensing benefits, not the privacy ones.

Redaction before the call

The cheapest compliance control is not sending the data. Order of preference:

1. Don't include it. Most prompts carry more record fields than the task needs. Trim to the columns the model actually reasons over.
2. Tokenize reversibly. Replace identifiers with stable placeholders (<CUSTOMER_7f2a>), keep the mapping in your own store, and rehydrate the response. This preserves coreference across a long document; naive masking destroys it.
3. Detect and redact. Pattern matching catches structured identifiers — card numbers, national IDs, emails. Free-text names, addresses, and rare diagnoses need an NER model, and it will miss things. Treat detection-based redaction as risk reduction, never as a basis for claiming data was de-identified under GDPR or HIPAA.

Masked context degrades reasoning where identity matters, so measure the quality cost. If a small local model does extraction on-premises and only aggregate results reach the frontier model, that split is usually cheaper as well as safer — see the cost calculator for the two-model arithmetic.

Audit logging

Build your own log; do not rely on the provider's. Per request: timestamp, authenticated end user, purpose, model, provider, region, provider request ID, token counts, prompt hash, tool calls, redaction policy version. Store hashes by default and full content only where a retention justification exists — your own logs carry the same DSAR and breach obligations as any other store. The request ID is the join key for a later investigation, so capture it on success too.

Compliance frames worth knowing

Which document answers which question — not legal advice.

• SOC 2 Type II — an audited report on security controls over a period. Read it, including the exceptions and complementary user entity controls; a "SOC 2 compliant" badge without a report is not evidence.
• ISO/IEC 27001 — certifiable information security management system. Check the Statement of Applicability and the scope: it may cover the corporate network, not the inference service.
• ISO/IEC 42001 — the AI management system standard. It certifies that a governance process exists (risk and impact assessment, lifecycle controls), not that any model is safe or accurate.
• GDPR — you are typically the controller, the API provider a processor under Article 28. You need a DPA, a subprocessor list, a transfer mechanism for non-EEA processing (SCCs plus a transfer impact assessment), and a documented lawful basis. Prompts about identifiable people are personal data, and so are outputs about them.
• HIPAA — a BAA is available from the major providers, usually only on specific plans and endpoints. Check endpoint-level coverage, not just "the vendor signs BAAs."
• EU AI Act — obligations attach to roles (provider, deployer, importer, distributor) and scale with risk tier (prohibited, high-risk, limited-risk transparency, minimal). Most teams calling an API are deployers: duties centered on human oversight, input data relevance, log retention, and informing affected people. You become a provider by putting a system on the market under your own name or substantially modifying one. Prohibitions and AI-literacy duties applied from 2 February 2025, GPAI model rules from 2 August 2025, most remaining provisions from 2 August 2026; high-risk obligations land 2 December 2027 (Annex III) and 2 August 2028 (Annex I). Confirm the dates — this schedule has moved before.

Due-diligence checklist

Ask for written answers, not sales calls:

• Do you train on our inputs or outputs under any tier or feature? What is the opt-in mechanism, and can it be disabled org-wide?
• Default retention for prompts, completions, files, feedback submissions, and embeddings — each separately.
• Is ZDR available on our plan, for which endpoints, and what do we lose?
• Which storage regions, which processing regions, and what is the in-region fallback behavior?
• Subprocessor list, notice period for changes, and right to object in writing.
• The paper: SOC 2 Type II report, ISO 27001 scope, DPA, SCCs, TIA, BAA endpoint coverage.
• When do humans read customer content — which staff, under what access controls, and is it logged?
• Deletion on request and on termination: what SLA, what proof? Incident notification in how many hours, to whom?
• If this is a gateway or aggregator, which downstream providers can serve our traffic, and can we restrict the set?

What people get wrong

• Assuming vendor uniformity. The consumer app, paid API, free API tier, and cloud-marketplace deployment of one model are four distinct contracts.
• Treating ZDR as free. Teams sign ZDR, discover conversation state, batch retrieval, or file search no longer works, and rebuild it themselves — badly.
• Assuming a cloud reseller inherits the model vendor's posture. Deploying a third-party model in your cloud tenancy changes the processor, the subprocessors, and sometimes the model versions and safety filters you get.
• Ignoring embeddings. Vectors are derived from source text and partially invertible. An embedding store built from customer records is a store of customer data, with the same deletion duties.
• Forgetting tool outputs and RAG context. Retrieval often injects more sensitive data than the user's message did, and redaction applied only to user input misses it.
• Treating feedback buttons as UI. A thumbs-down can ship an entire conversation into a training-eligible store on multi-year retention. Decide whether that control stays on.
• Logging full prompts forever "for debugging." Self-inflicted, and the most likely source of your own breach.
• Blocking on a compliance review that never scopes the workload. A prompt with no personal data and a prompt with patient notes do not need the same controls. Classify workloads first; the model index and comparison tool show which providers serve each class under the terms you need.