If your AI product ever touches a patient's health information, HIPAA is not optional reading. It is a federal law with real enforcement behind it, and building an AI feature does not create an exception to it. This is a plain-language walk through what HIPAA actually requires, why it exists, and specifically what changes when the thing handling that data is an LLM-based product instead of a traditional application.
What HIPAA actually is
HIPAA is the Health Insurance Portability and Accountability Act, a US federal law enforced by HHS (the Department of Health and Human Services). Two parts of it matter most for software: the Security Rule, which sets administrative, physical, and technical safeguards for electronic health information, and the Privacy Rule, which controls who can access, use, or disclose that information and how much of it they can touch for a given purpose.
HIPAA applies to two kinds of organizations. A covered entity is a health care provider, health plan, or health care clearinghouse that transmits health information electronically, think a clinic, an insurer, or a billing service. A business associate is any person or company that creates, receives, maintains, or transmits protected health information (PHI, meaning individually identifiable health information) on behalf of a covered entity, including subcontractors of a business associate. This second category is the one that catches software vendors off guard, and it is exactly the category an AI vendor, or the LLM API provider behind that vendor's product, can fall into the moment PHI passes through their systems. The regulatory definition sits at 45 CFR 160.103.
Why HIPAA exists
Before HIPAA, there was no consistent federal floor for how health information got handled. Medical records could move between insurers, employers, and other parties with little control over who saw what, and patients had no reliable way to know or object. HIPAA exists to address a specific, real harm: health information is uniquely sensitive, and its exposure carries risks (discrimination, embarrassment, financial harm, loss of trust in a care relationship) that go beyond an ordinary data breach. The Privacy Rule's minimum necessary standard exists for the same underlying reason: even inside a legitimate healthcare relationship, more access than needed is still a risk.
Why a company follows it
Two reasons, and only one of them is about avoiding a penalty. HHS enforces HIPAA and can investigate complaints and breaches, so there is real legal exposure for a covered entity or business associate that mishandles PHI. But the more immediate reason most companies care is trust: a healthcare buyer will not sign a contract with a vendor that cannot explain, clearly, how its product handles PHI, whether a business associate agreement is available, and what happens to data sent into any AI component. A weak answer to that question ends the sales conversation before it starts.
One thing worth being direct about: there is no such thing as "HIPAA certification" from HHS. HHS does not certify organizations, products, or vendors as HIPAA compliant, full stop. A surprising number of vendors advertise "HIPAA certified" anyway. That phrase alone is worth treating as a small red flag about how carefully a vendor understands the rule it is claiming to meet.
What specifically needs attention when building AI products under HIPAA
PHI in prompts and logs
Every place PHI can end up needs to be accounted for, not just the primary database. If a support agent's prompt includes a patient's name, diagnosis, or treatment history, that prompt is PHI the moment it is assembled, whether it is stored anywhere or not. Application logs, LLM request and response logs, error-tracking tools, and any analytics pipeline that captures user input are all places PHI quietly ends up if nobody designs against it. Consider a small healthcare scheduling startup building an AI intake assistant: if the assistant's conversation transcripts are shipped to a third-party logging service by default, that service has just become part of the compliance picture, whether anyone planned for it or not.
Business associate agreements with model vendors
If PHI reaches a model API, a logging pipeline, or a vector store operated by anyone other than the covered entity itself, that operator is very likely acting as a business associate (or a subcontractor of one), and a signed business associate agreement (a BAA, the contract that extends HIPAA obligations to that vendor) needs to be in place before PHI ever reaches those systems, not after. Not every LLM API provider offers a BAA, and not every plan tier of providers that do offer one includes it. This has to be checked directly with the provider and confirmed in writing before PHI is sent to any endpoint, retrieval index, or logging destination.
The minimum necessary standard applied to what an agent retrieves
The Privacy Rule's minimum necessary standard says a covered entity or business associate should access, use, or disclose only the minimum PHI needed for the task at hand. For a traditional application this usually means role-based access control. For an AI agent, it means something more specific: what records can the agent actually query, and how broad is that query allowed to be. An agent that answers "what is this patient's next appointment" should not have standing access to that patient's entire chart, a diagnosis history, and every other patient's records in the same table, even if the underlying database technically allows the query. Scoping this at the system level, not the prompt level, is the difference between a design that holds up and one that depends on the model always behaving as instructed.
Administrative, physical, and technical safeguards
The Security Rule's three safeguard categories still apply in full to an AI-enabled product. Technical safeguards cover things like access controls, audit logging of who accessed what PHI and when, and encryption in transit and at rest. Administrative safeguards cover workforce training and a designated security official. Physical safeguards cover facility and device access. None of this goes away because a language model is part of the stack; if anything, audit logging becomes more important, because an AI agent that touches PHI needs a trail showing what it retrieved and why, not just what a human user did.
What GreyScript AI does, and does not, do here
GreyScript AI builds AI systems designed to align with HIPAA's Security Rule and Privacy Rule: scoping what an agent can retrieve, keeping PHI out of logging paths that were not built for it, and structuring the system so a business associate agreement with the relevant vendors is a real, checkable thing rather than an assumption. We do not issue certifications or audit reports, and as covered above, there is no official HIPAA certification to issue in the first place. Where a formal audit, a legal compliance determination, or a business associate agreement needs to be signed, that stays yours to commission, typically through counsel and the vendors in question directly. See our AI governance and security service for how this fits into a build.
This is closely related to the broader question of building AI for regulated industries generally. If healthcare is one of several constrained domains you are navigating, our piece on building AI for compliance-heavy industries covers how the project shape changes across healthcare, fintech, and insurance.