Science

New legal framework determines who’s to blame for a rogue AI

New legal framework determines who’s to blame for a rogue AI


An AI system can produce a harmful answer that nobody explicitly told it to give. That leaves an uncomfortable gap between the damage someone experiences and the decisions that made it possible. Blaming the software offers little help to a person seeking accountability.

Martin Hilbert, a communication professor at the University of California, Davis, proposes following those decisions back through the people and organizations involved. His paper in the Proceedings of the National Academy of Sciences argues that existing legal principles can organize responsibility across an AI system’s life. It is a proposed framework, rather than a new law or an experimental finding.

The approach connects concerns about cybersecurity risks and harmful chatbot design with a practical question: which choices contributed to a foreseeable injury? Hilbert argues that answering it does not require deciding whether a machine thinks, speaks consciously or deserves legal personhood.

Courts already have ways to address defective products, negligent conduct and responsibility for delegated tasks. AI complicates their application because several companies and users may shape the same system. Its developer chooses how it learns, another business chooses where it operates, and a user supplies the immediate request. Being unable to predict every output would not, in Hilbert’s view, erase responsibility for known kinds of failure.

Speech protections and platform immunity add further complications. Section 230 generally concerns liability for information supplied by someone else, while generative systems produce new outputs. How particular claims fit those protections remains a legal question, rather than something the framework settles by itself.

The distinction matters wherever automated judgments affect people. An allegation about unsafe design may raise different issues from an allegation about publishing another person’s words. Neither analysis automatically requires recognizing the machine as a person with rights.

“Courts adapt tools to regulate earlier technologies, but many of these tools were never designed to address AI,” Hilbert said. His proposal seeks a common organizing principle across those tools, including disputes involving algorithmic bias. It focuses attention on who made consequential choices and what followed from them.

Following five layers of human choices

The first layer is the objective: what the system should optimize. A developer might reward accurate predictions or successful completion of a task. Hilbert treats that choice as the starting point because a learning system needs an objective against which its performance can be assessed. Choosing success criteria can create incentives that differ from what the public expects the system to do.

Next come training and alignment, including data selection, architecture and feedback that shape behavior. These decisions influence the range of responses a model can produce, even when nobody selects its exact next sentence. Work on model design illustrates how objectives and supporting components can shape performance.

The third layer concerns deployment: the intended audience, application, permissions and safeguards. The fourth covers the user’s instructions and attempts to direct or manipulate the system. Even increasingly autonomous systems operate within boundaries established through such choices.

The fifth layer is ongoing care after deployment, including monitoring failures and responding when problems become apparent. “Every AI, without exception, starts with goals that a human defines, and every subsequent step in the AI’s development involves a chain of deliberate human choices,” Hilbert said. His framework asks how those choices connect to responsibility.

Three questions turn the map into an assessment

Simply appearing somewhere in that chain would not establish liability. Hilbert proposes asking three questions at each layer: whose decision was involved, whether it caused foreseeable harm, and how responsibility should be divided. The questions concern evidence and conduct, not a presumption that every contributor shares equal blame.

The first step identifies the relevant actor and decision. The second examines causation and foreseeability, including whether a reasonable alternative could have reduced the danger. Someone need not have wanted an injury for their choices to warrant scrutiny, but the connection still needs establishing. Consider a chatbot marketed for health advice: its intended use would matter when assessing whether dangerous misinformation was foreseeable.

For example, a user who deliberately bypasses reasonable safeguards presents a different case from someone harmed during ordinary use. A company that ignored known weaknesses may also bear responsibility. The framework offers a way to examine both contributions without assuming one cancels the other.

Finally, responsibility must be apportioned using the applicable legal rules. Hilbert emphasizes that this is not a calculation that produces precise fault percentages from model behavior. Efforts to explain failures can inform the evidence, while courts still need to assess duty, causation and the circumstances of each claim.

Shared systems create problems of evidence and compensation

The paper identifies four gaps that could make this approach difficult to enforce. The first involves supply chains: one company supplies data, another trains a model, and others modify or deploy it. Separating their contributions may prove especially difficult when several decisions jointly produce an injury. For an injured claimant, the difficulty is obtaining enough information to show how those contributions fit together.

Hilbert draws an analogy with Superfund environmental liability, which addresses responsibility for hazardous-waste cleanup. Under that law, a party can sometimes face the whole cleanup bill when harm cannot be divided. That is a model for thinking about shared responsibility, rather than a rule that automatically applies to AI.

Vaccine compensation and pooled insurance offer other examples of institutions addressing uncertain causation and distributing costs. These comparisons concern mechanisms for handling claims, not evidence that AI harms follow the same causes. Adopting comparable arrangements would involve policy choices beyond describing the development chain.

The second gap concerns access to evidence inside opaque systems. Hilbert proposes a rebuttable presumption that could require companies to demonstrate adequate care when foreseeable harm occurs. Records of safety testing could help assess that claim, but the proposed burden shift is not a universal rule already governing AI lawsuits.

Responsibility continues after the system goes live

The third gap is change over time. Later modifications can alter a system’s behavior, including changes that remove safeguards. The framework would examine those later choices and the obligation to respond to emerging problems, rather than assigning everything permanently to the original developer.

That makes monitoring important alongside initial testing. Research into AI behavior and explainable decisions can contribute to understanding systems in use. Hilbert’s concern is what organizations do when evidence of danger becomes available.

The fourth gap is speed: litigation can take years while systems operate at enormous scale. Hilbert argues for stronger public technical capacity and links between evaluation and enforcement. The accountability framework draws on existing law, while those institutional changes remain proposals requiring their own legal and policy decisions.

“It takes a village to raise an AI, and everyone in that village bears a duty of care,” Hilbert said. The practical task is to identify each participant’s decisions and assess their consequences. An unexpected answer would begin that inquiry, rather than end it.

Dig deeper into AI accountability and risk

These resources explain technical risks and existing approaches to enforcement, shared liability and compensation.

  1. Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations: Defines attacks across the AI life cycle and describes ways to mitigate their consequences, providing context for evaluating foreseeable security risks. (NIST, March 2025)
  2. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile: Offers voluntary guidance for identifying and managing generative AI risks throughout development, deployment and evaluation. (NIST, July 2024)
  3. FTC Chair Khan and Officials from DOJ, CFPB and EEOC Release Joint Statement on AI: Explains the agencies’ position that existing consumer-protection, competition and civil-rights authorities apply to automated systems. (Federal Trade Commission, April 2023)
  4. Superfund Liability: Describes strict, joint and several liability for hazardous-waste cleanup, including circumstances in which one responsible party may face the full cleanup cost. (Environmental Protection Agency)
  5. About the National Vaccine Injury Compensation Program: Explains a statutory, no-fault alternative to traditional litigation, including compensation through settlements without a finding that a vaccine caused an injury. (Health Resources and Services Administration)

Research findings are available online in the journal PNAS.






Source link