Clinical AI Regulation in the US Depends on What a Tool Is Meant to Do

Clinical AI regulation in the United States doesn’t come from a single law or agency. A sepsis alert in an electronic health record, an algorithm that flags a suspected brain bleed on a CT scan, and an ambient scribe that drafts a visit note can all be described as clinical AI, yet each may fall under a different set of rules. For hospitals, clinicians, and developers, the first question is what a tool is intended to do, who relies on its output, and how quickly a decision must be made. Whether it uses AI is secondary.

This explainer walks through the main federal frameworks shaping clinical AI today: the Food and Drug Administration’s device authority, its guidance on clinical decision support software, the mechanism for updating AI models after authorization, and the health IT transparency rules overseen by the Department of Health and Human Services. It is educational context, not legal or medical advice, and organizations should confirm how any rule applies to a specific product with qualified regulatory counsel.

Why Intended Use Decides Which Rules Apply

The FDA regulates medical devices, and software can be a device when it is intended to diagnose, treat, mitigate or prevent disease. Regulators infer that intent from labeling, marketing claims, and how the product is designed to be used, not from the underlying technique. Regulators treat a gradient-boosted model and a large language model the same way if they make the same kind of claim to the same kind of user.

Congress carved certain software functions out of the device definition in section 520(o) of the Federal Food, Drug, and Cosmetic Act. One category is clinical decision support, or CDS, which helps a health care professional decide without replacing that professional’s judgment. Administrative tools, such as scheduling or billing software, fall outside device oversight for different reasons. The result is a sorting exercise: the same hospital may run AI tools that are FDA-authorized devices, tools that qualify as non-device CDS, and purely operational tools.

The distinction affects buyers as much as developers. An FDA-cleared or authorized product has undergone premarket review of its claims. A non-device CDS tool has not, which doesn’t make it unsafe, but it shifts more of the validation burden onto the organization that deploys it.

The Four Criteria for Non-Device Clinical Decision Support

Federal law sets out four criteria that a software function must meet together to be excluded from device regulation as CDS. If it fails to meet any one, the function is generally treated as a device. In plain terms, the software must meet all of the following:

1. Not acquire, process or analyze a medical image, a signal from an in vitro diagnostic device or a pattern or signal from a signal acquisition system.
2. Display, analyze or print medical information about a patient or other medical information, such as clinical guidelines or peer-reviewed studies.
3. Support or provide recommendations to a health care professional about prevention, diagnosis or treatment.
4. Enable that professional to review the basis for the recommendation independently, so the professional does not rely primarily on the software to make the decision.

The first criterion removes a large share of clinical AI from the CDS exclusion. Imaging algorithms, ECG waveform tools, and software that analyzes continuous physiological signals generally process images or signals. As a result, regulators typically treat them as devices regardless of how they present their output.

The fourth criterion is often the hardest for machine learning to meet. If a clinician cannot see the inputs, logic, and supporting evidence well enough to reach their own conclusion, the software is doing more than supporting a decision. Opaque models that produce a risk score without explanation can struggle with this.

What Changed in the FDA’s January 2026 CDS Guidance

On January 6, 2026, the FDA issued a revised final guidance on clinical decision support software, replacing the version it published in 2022. Law firms that reviewed the document, including Covington and Cooley, described the changes as incremental rather than a rewrite. The FDA did not change its statutory interpretation of the four criteria, but it adjusted how it applies them in a few areas relevant to AI developers.

Single Recommendations

The earlier guidance viewed software that gives one specific diagnostic or treatment output, rather than a list of options, as generally failing the third criterion. The revised guidance keeps that position as a matter of interpretation but adds an enforcement policy. Where only one recommendation is clinically appropriate, and the software otherwise meets the criteria, the FDA says it intends to exercise enforcement discretion. Commentators noted that the guidance does not define how to judge when a single option is clinically appropriate, which leaves room for disagreement.

Time-Critical Decisions

The revision moved the discussion of time-critical decision-making from the third criterion to the fourth. The logic is that when a clinician must act quickly, there is little opportunity to review the basis of a recommendation, so the software is more likely to be relied on directly. Stroke alerts, sepsis alerts and hypoglycemia warnings from continuous glucose monitors appear among the FDA’s examples of functions that remain devices, in part because they support urgent decisions.

Automation Bias

The guidance defines automation bias as the propensity to overrely on an automated system’s suggestion. The FDA uses that concept to interpret whether a clinician can genuinely review a recommendation independently, and it observes that automation bias increases in situations requiring urgent action. For hospital AI committees, the point is that a tool that technically shows its reasoning may still encourage overreliance if it is embedded in a high-pressure workflow.

How AI Devices Reach the Market and How Models Get Updated

When clinical AI is a device, it reaches the US market through one of the FDA’s premarket pathways. The most common are premarket notification, known as 510(k), which shows a device is substantially equivalent to one already on the market; the De Novo pathway for novel low- to moderate-risk devices; and premarket approval, or PMA, for the highest-risk devices. Each authorization covers a specific intended use and a specific version of the product.

That last point creates a problem for machine learning. Models can improve with new data, but significant changes to an authorized device can require a new submission. To address this, Congress added section 515C to the Federal Food, Drug, and Cosmetic Act through the Food and Drug Omnibus Reform Act of 2022, creating a formal basis for predetermined change control plans, or PCCPs.

The FDA finalized its guidance on PCCPs for AI-enabled device software functions in a notice published in the Federal Register on December 4, 2024. Under that guidance, a PCCP describes the planned modifications, sets out the methodology for developing, validating and implementing them and includes an assessment of their impact. If the FDA authorizes the plan as part of the original submission, the manufacturer can make the described changes without a new marketing submission for each one, as long as it follows the plan.

An analysis of the final guidance by the law firm Ropes & Gray pointed to several operational details. The scope expanded from machine learning devices to AI-enabled devices more broadly. The guidance calls for guardrails on automatic updates and expected update frequency, recommends labeling that tells users when modifications are made, and calls for public summaries describing the plan. For 510(k) devices, PCCPs are available through the traditional and abbreviated pathways, not the Special 510(k).

For hospitals, it’s worth asking directly about a PCCP. A vendor whose authorized product includes one can update its model within defined limits, and a buyer should understand those limits, how changes will be communicated, and how local performance monitoring will detect any drift.

Tracking What the FDA Has Authorized

The FDA maintains a public list of AI-enabled medical devices authorized for marketing in the United States. According to the agency’s page, devices are identified primarily by AI-related terms in the summary descriptions of their marketing authorizations or in their device classification, using terms from the FDA’s Digital Health and AI Glossary. The list is updated periodically rather than on a fixed schedule, and the FDA states that it is not comprehensive.

The list is a reasonable starting point for due diligence. A buyer can check whether a product, or the specific indication being marketed, appears in it and then read the underlying decision summary. Some entries note that the authorization includes a PCCP. The FDA page also says the agency will explore ways to identify and tag devices that incorporate foundation models, including large language models, and it encourages sponsors to describe such features in their public summaries.

Absence from the list does not by itself mean a product is unlawful. It may be non-device CDS, an administrative tool or a device whose summary does not use the search terms. That ambiguity is why buyers should ask vendors for a clear statement of regulatory status and the basis for it. Readers following new authorizations can find ongoing coverage in the TechAmerica.ai Biotech and Health section.

Health IT Transparency Rules and Their Uncertain Future

Not all clinical AI arrives as a standalone product. Much of it is built into electronic health record systems, and HHS applies a separate set of rules. The Office of the National Coordinator for Health Information Technology, now part of the Assistant Secretary for Technology Policy (ASTP/ONC), runs a voluntary certification program for health IT that most major EHR vendors participate in.

The HTI-1 final rule, published in January 2024, replaced the program’s older CDS criterion with a new criterion for decision support interventions, or DSIs. It defines predictive DSIs as technology that supports decision-making based on algorithms or models that derive relationships from training data. Certified developers that supply predictive DSIs must make 31 source attributes available to users, covering items such as intended use, target population, known risks, training data and validation. Evidence-based DSIs carry 13 source attributes. Starting January 1, 2025, developers with certified DSI functionality had to meet the related maintenance requirements.

HTI-1 also requires those developers to apply intervention risk management practices in three areas: risk analysis, risk mitigation and governance. These requirements apply to the certified developer and do not extend to predictive models that hospitals or other parties build themselves.

The future of these source attribute requirements, sometimes described as model cards, is unsettled. In a proposed rule published on December 29, 2025, ASTP/ONC proposed eliminating 34 of 60 existing certification criteria and revising others, including removing the requirement for developers to supply source attribute information. On February 27, 2026, California Attorney General Rob Bonta submitted a comment letter opposing that change. Before anyone relies on it, check whether the requirement survives in a final rule against current ASTP/ONC publications.

Where the Regulatory Gaps Sit

Several common clinical AI uses fall between these frameworks. Ambient documentation tools, which record clinical conversations and draft notes, are generally positioned as administrative aids rather than diagnostic tools. That positioning can keep them outside device review, even as products expand into suggesting orders or codes. TechAmerica.ai reports that AI medical scribes are moving beyond note-taking, making intended-use questions harder.

Models that a health system builds for its own use also sit in a gray zone. HTI-1’s certified developer requirements don’t cover them, and whether FDA oversight applies depends on their function and deployment. Internal governance committees often carry most of the responsibility for validating and monitoring such tools.

Generative AI raises its own questions. A large language model that summarizes chart information for a clinician might fit within CDS if it meets all four criteria, while one that produces specific treatment directions or interprets images would not. The FDA’s stated interest in tagging devices that use foundation models suggests the agency is watching this area closely, and its published agenda has signaled further work on AI policy. TechAmerica.ai has covered how the FDA is prioritizing AI and surgical robot guidance in its planning.

Other federal and state rules can also apply. Health privacy law governs the data that trains and feeds these systems, civil rights rules address discrimination in patient care, and some states have passed their own requirements for AI in health care. Those areas are beyond the scope of this explainer, but they belong on any compliance checklist.

Questions Hospitals and Clinicians Should Ask

For organizations evaluating a clinical AI tool, the regulatory picture raises several questions. None of them replaces specialist legal and clinical review, but they help separate products with a clear evidence base from those that rely on marketing language.

• Regulatory status: Is the product an FDA-authorized device, non-device CDS or an administrative tool, and what is the documented reasoning for that position?
• Authorized scope: If it is a device, what exact indication, patient population, and user does the authorization cover, and does the planned use match it?
• Update plan: Does the authorization include a PCCP, and how will the vendor notify users when the model changes?
• Transparency: Can clinicians see the inputs and basis for each recommendation, and what source attribute information does the EHR vendor provide?
• Local validation: How will the organization test performance on its own patient population before go-live and monitor it afterward?
• Workflow risk: Is the tool used in time-pressured settings where automation bias is more likely, and what safeguards exist?

Clinicians have a role here too. Whether a tool is meant to inform judgment or drive a specific action changes how much weight its output should carry. Clear labeling and training help, but so does a culture that treats questioning an algorithm as normal practice.

Clinical AI regulation will keep moving. The FDA has updated its CDS guidance, formalized a path for model updates and started thinking about foundation models, while HHS is reconsidering the transparency rules it adopted only two years ago. For readers, the lesson that will last is to start with intended use, check the primary sources and treat every vendor claim about regulatory status as something to verify rather than assume.