HIPAA and LLMs: You Are Scoping It Wrong

I sat in a meeting where a Compliance Officer told an Engineering Director: "We can't use an LLM because we can't control what it outputs."

This is a fundamental misunderstanding of how HIPAA applies to software. HIPAA doesn't demand that your software be deterministic. It demands that you protect Protected Health Information (PHI).

If you are building healthcare AI, you are stuck in one of two modes:

1. The Wild West: "Just sign a BAA with OpenAI and send them everything." (Dangerous)
2. The Paralysis: "We can't use AI until we build our own on-prem GPU cluster." (Expensive)

There is a middle path. It's called Data Minimization, and it's how the best teams ship.

The "Minimum Necessary" Rule

HIPAA has a rule called Minimum Necessary. It means: Don't access data you don't need.

Most RAG (Retrieval-Augmented Generation) pipelines violate this by default.

• User Query: "Does the patient have a history of diabetes?"
• Naive RAG: Fetches the entire electronic health record (EHR) JSON blob.
• The Leak: That JSON blob contains their address, social security number, and mental health notes. You just sent all of that to an LLM for a simple "Yes/No" question.

Fix: Create a "Clinical Summary" projection of your data. Only index medical facts. Do not index demographics (PII) in the same document store.

The BAA Fallacy

Signing a Business Associate Agreement (BAA) with OpenAI/Anthropic/AWS is necessary, but it is not a "Get Out of Jail Free" card.

If you send a prompt containing "Patient John Doe (SSN: 123-45-6789) has..." to a model, you are creating a Permanent Record of that PHI in the model's logs (even if they don't train on it). If that account is compromised, you have a breach.

Fix: Redact identifiers before they leave your VPC. Use a specialized NER (Named Entity Recognition) model to swap names for placeholders:

"Patient [NAME_1] (SSN: [REDACTED]) has..."

The LLM is just as smart. But now the data is toxic waste.

The "Audit Trail" Trap

Compliance auditors ask: "Who asked what?" If your answer is "It's in the application logs," you are failing.

AI is conversational. A user might ask 5 questions to get 1 answer. You need a Semantic Audit Log:

• Not just the raw HTTP request.
• The Intent (e.g., "Querying Patient History").
• The Context retrieved (e.g., "Accessed Document #451").
• The Outcome (e.g., "Provided diagnosis summary").

Conclusion

HIPAA compliance isn't about buying a tool. It's about architecture. If you treat the LLM as an untrusted public API (even with a BAA), you will build a secure system by default.