TL;DR: A signed BAA covers the vendor's data storage and transmission. It does not cover what happens when PHI enters a model, gets processed by attention layers, and exits as a synthesized response. The inference layer is a structural compliance gap that standard BAAs cannot address. Closing it requires new contract language, technical controls at the prompt boundary, and inference auditing that proves what the model saw, generated, and retained.

Key Takeaways: - Standard BAAs were designed for deterministic data processors like databases and file stores, not for non-deterministic model inference on PHI. - Four gaps sit inside the inference stack: training data lineage, prompt and response logging, model output as PHI, and subprocessor model dependencies. - Defense-in-depth means prompt redaction, isolated inference environments, ephemeral logging, and continuous audit instrumentation, layered together.

Your AI vendor signed your BAA. You checked the box. Your compliance team moved on. But the inference layer was never in the agreement.

The inference layer is the place where patient data actually gets processed by the model, and most CISO's have not mapped what that means for their compliance posture.

Most CTOs treat the BAA as a finished compliance artifact. Once the document is signed, AI procurement is done. That assumption is wrong. BAAs were designed for data storage and transmission. They were written for database vendors, cloud storage providers, and transmission channels with predictable inputs and outputs.

They were not written for a non-deterministic model. A model that actively processes PHI through learned parameters and returns a synthesized response is outside the BAA's original design. For teams building or buying AI compliance programs, this is where audit findings start.

Signing a BAA does not mean the model is compliant. It means the vendor's storage and transmission practices are. Those are two different things. The inference layer is where patient data is actively processed, transformed, and sometimes memorized.

The BAA's standard "permitted uses" and "safeguards" clauses were never designed to govern a transformer architecture. The inference layer is not a vague concept your legal team can hand-wave away. It is a specific technical stack with specific failure modes, and almost no vendor BAA addresses it. So what actually happens inside it?

Inference is real-time computation. A prompt containing PHI enters the model. The attention mechanism processes it against learned weights. An output is generated, token by token, and returned to the caller.

From the user's perspective, this looks like a single API call. Under the hood, it is a multi-stage pipeline with persistence risks at every step. For a deeper look at how transformers actually move tokens through inference, see Inside the AI Brain.

Consider the KV cache. During inference, transformers store key-value pairs in memory to speed up sequential token generation. These caches can retain fragments of prior inputs.

That creates accidental PHI persistence even after a session ends. If the cache is swapped, snapshotted, or leaked through a side channel, PHI exits the boundary your BAA was supposed to enforce.

Logging is the next risk. Inference generates traces, telemetry, and request logs that often contain full prompts and responses. Unredacted PHI sits in vendor infrastructure, indexed for debugging, and retained according to operational policies that have nothing to do with HIPAA.

The model itself, when fine-tuned or evaluated, may absorb your traffic. Many vendors reserve the right to learn from production prompts unless you negotiate an explicit opt-out. For organizations running AI/ML training pipelines on production data, this is a silent compliance leak. If the inference layer behaves this way, does the BAA actually cover any of it?

The HIPAA BAA template was built around data processors with predictable behavior. Databases store rows. File systems store objects. Transmission channels move bytes.

Each has well-understood inputs, outputs, and retention characteristics. The BAA's obligations (to safeguard, to report breaches, to limit disclosure) map cleanly onto that reality.

AI models break every one of those assumptions. They are non-deterministic: the same PHI prompt can produce different outputs across runs. They cannot guarantee what they will do with input data.

The BAA language covers "use" and "disclosure" of PHI. It does not address model behavior, output generation, or downstream retention in model weights. Those are ungoverned activities in most vendor agreements.

Vendors know this and have built their agreements accordingly. Many explicitly disclaim liability for model outputs and for any learning that occurs during inference. The fine print says the model "may produce inaccurate information" and that customer data "may be used to improve services." Your BAA did not negotiate those clauses.

They came in the standard MSA. For healthcare technology buyers, this is the blind spot that shows up in every post-deployment audit. So where exactly does the compliance gap live? It sits in four specific places inside the inference stack.

Gap 1: Training Data Lineage

Was the model trained on PHI? Most vendors will not tell you. Their model cards mention general datasets. Your BAA almost never asks. If the base model absorbed PHI during pre-training, every inference call inherits that risk. You cannot redact it away at the prompt layer. The contamination is baked into the weights.

Gap 2: Prompt and Response Logging

Inference logs frequently store raw prompts and outputs. These logs exist for debugging, observability, and model evaluation. If they are retained, backed up, or fed into an eval pipeline, PHI is being re-stored outside the BAA's scope. The vendor's log retention policy is now a HIPAA question, and your contract never addressed it. This is why AI/ML training pipelines that touch production traffic are a compliance landmine.

Gap 3: Model Output as PHI

The model can generate PHI that did not exist in the input. Synthesized patient summaries, diagnostic suggestions, and medication recommendations are all outputs derived from input. Who owns that output? Who is responsible if it is wrong? Standard BAAs do not address generated PHI because the template authors never imagined a system that creates new protected health information on the fly.

Gap 4: Subprocessor Model Dependencies

Your vendor may route inference to a third-party model API. OpenAI, Anthropic, open-source models hosted on another cloud, all of these are subprocessors. Your BAA with Vendor A does not cover Vendor B's inference layer. Vendor B's BAA, if one exists, is between Vendor B and Vendor A, not you. You sit at the end of a chain of contracts that does not bind you to the subprocessor.

Identifying the gaps is step one. Closing them requires contract language your legal team has probably never negotiated, and technical controls your engineering team has probably never built. So where do you start?

Open the BAA your vendor sent you. Read it top to bottom. If you cannot find the words "inference," "model output," or "prompt processing," the agreement was not written for AI. It was templated from a SaaS storage contract. That is your first red flag.

Five questions to ask before you sign: - Does the BAA explicitly mention "inference," "model output," and "prompt processing," or only "data storage" and "transmission"? - Is there a clause prohibiting the use of customer prompts or outputs for model training, fine-tuning, or evaluation? - Are all subprocessor models enumerated, and does the vendor warrant BAA-equivalent coverage for each one? - Is there a right to audit inference logs, and a contractual commitment to log redaction or ephemeral processing? - Can the model be deployed in a customer-controlled environment (VPC, on-prem) rather than a shared multi-tenant inference endpoint?

If the answer to the first question is "only storage and transmission," you are not buying a compliant AI system. You are buying a compliance liability with a signed cover page. The contract review is where the work starts, not where it ends.

Contracts alone won't close the gap. The engineering layer has to enforce what the contract cannot guarantee. Four controls, layered together, will close the inference gap:

PHI redaction at the prompt layer. Strip identifiers before they reach the model. Inject synthetic tokens for inference, then reconstruct the response downstream with a local mapping table. The model never sees real PHI, and your output remains useful.

Isolated inference environments. Deploy models inside your own VPC or on-prem. Now the BAA covers your infrastructure, not the vendor's shared multi-tenant cluster. You control the network boundary, the encryption keys, and the access logs.

Ephemeral inference. Configure vendors to disable logging, set TTLs to zero, and prevent any retention of prompts or outputs. Most enterprise model APIs support a "no-log" or "zero-retention" mode. Turn it on. Verify it with a test prompt and a log search.

Continuous inference auditing. Instrument every inference call to capture what entered the model, what came out, and where it went. You need to prove compliance during an audit, not reconstruct it from vendor logs you do not control.

When you close the inference gap, the conversation with your CISO, your auditors, and your clinical stakeholders changes entirely.

Compliance stops being a checkbox. Auditors stop finding inference-layer gaps because you have engineered them shut. Your clinical and compliance stakeholders trust the system because they can see the controls, not just the contract.

Deployment timelines compress because legal review no longer stalls on "what happens at inference." The architecture was defensible from day one.

This is the difference between systems that survive a five-year production run and systems that get ripped out after a failed audit. When healthcare technology platforms are built with inference-layer discipline from the start, the long-term economics flip. That is the compounding return of doing the hard work upfront. The questions your auditor will ask come next.

Q: Does a BAA cover AI inference under HIPAA?

A: No, not in any standard form. A BAA covers the vendor's storage, transmission, and processing of PHI. Standard agreements do not address model behavior, prompt logging, or inference-time computation. You need explicit contractual addenda to cover the inference layer.

Q: What is the inference layer in healthcare AI?

A: The inference layer is where a trained model actively processes input data to generate outputs. In healthcare AI, this is where patient data enters the model, gets transformed by attention mechanisms and learned weights, and produces a response. It is the highest-risk point in the AI stack for PHI exposure.

Q: Can a vendor use my prompts to train their model under a BAA?

A: Only if the BAA explicitly permits it, and most do not. Many vendors reserve the right to use customer traffic for model improvement unless you negotiate an opt-out. Always require an explicit clause prohibiting use of your prompts or outputs for training, fine-tuning, or evaluation.

Q: How do I audit my AI vendor for inference-layer compliance?

A: Request their data flow diagram for inference traffic. Ask for their logging and retention policies for prompts and outputs. Get their subprocessor list, including all third-party model APIs. Then ask for contractual commitments on training data usage. If they cannot produce these, the inference layer is ungoverned.

Q: Is self-hosted AI inference more HIPAA-compliant than cloud APIs?

A: Generally yes, because the BAA covers your own infrastructure rather than a shared vendor environment. Self-hosted inference lets you control logging, retention, redaction, and audit trails directly. The trade-off is operational complexity and the need for in-house ML infrastructure expertise.

Map your inference layer before your next audit maps it for you.

Research and references cited in this article:

• Healthcare AI Deployment: Compliance Through ...
• AI Review for Business Associate Agreements (BAAs)
• Business Associate Agreement (BAA) for Healthcare AI | Hello
• Business Associate Agreements for HIPAA Compliant AI in ...
• HIPAA Business Associate Agreements for AI Vendors: The Clauses That Actually Matter When PHI Reaches an LLM | DeepInspect
• HIPAA-Compliant AI: Why a BAA Alone Is Not ... | ibl.ai Blog
• Why Traditional Third-Party Risk Management Fails in the ...
• “HIPAA Compliant” Is Not a Certification: How Healthcare Organizations Must Vet AI Vendors That Overstate Their Compliance | Fisher Phillips LLP
• AI Business Associate Agreements (BAAs) in 2026: Which Vendors Will Sign One, and What That Actually Covers | The AI Career Lab
• How To Use AI While Staying HIPAA Compliant
• Risks for Healthcare Organizations Using AI
• Healthcare AI: Specific Risks & Challenges in Compliance