Who is accountable when AI breaks the law?

Imagine a future where an AI agent incorporates a company, files an annual return, renews a licence, submits a regulatory declaration or negotiates a commercial contract without a human ever touching the keyboard.

That future is closer than many people realise.

Today, organisations are already using AI to review compliance obligations, prepare regulatory filings, draft legal documents, analyse due diligence information and interact with digital government services. The next generation of AI agents is moving beyond generating content to performing tasks autonomously, often with increasing levels of authority delegated by the people and organisations that use them.

As AI becomes more capable, one question becomes impossible to ignore:

When an AI agent acts on behalf of a business, who is ultimately accountable if something goes wrong?

If an AI submits incorrect information to a corporate registry, breaches regulatory obligations, infringes intellectual property, commits fraud, or makes a decision outside the authority it was given, where does responsibility lie?

The AI? The organisation that deployed it? Or the individual who authorised it?

These are no longer hypothetical questions. Governments around the world are actively exploring how existing legal frameworks should evolve to govern autonomous systems. Yet while legislation continues to develop, many jurisdictions still lack the operational infrastructure needed to support accountability in an AI-enabled world.

That is where registries may have an important role to play.

Not because they can answer every legal question, but because they have spent decades solving one of government's most enduring challenges: establishing trusted, authoritative records that enable accountability, legal certainty and public confidence.

AI is moving from tool to participant

For years, AI has largely been viewed as a productivity tool, helping people write documents, analyse information or answer questions, but that distinction is beginning to disappear.

Businesses are increasingly using AI to draft company filings, review AML and KYC documentation, prepare procurement responses, assist with tax submissions and interact with government services. AI agents are also beginning to execute workflows across multiple systems with minimal human intervention.

People are no longer asking AI for advice, they are asking it to act on their behalf. That changes the governance challenge completely.

The critical question is no longer simply whether an AI made the right decision. It is whether the AI was authorised to make that decision in the first place; and if it was, that presents the following questions:

• Who delegated that authority?

• What actions was the AI permitted to perform?

• How can regulators verify who ultimately authorised the action?

These questions will feel familiar to registry professionals.

Registries already manage concepts such as legal authority, authorised representatives, directors, officers, beneficial ownership and delegated powers. AI introduces a new participant into that ecosystem, but many of the governance principles remain remarkably similar.

Accountability is about more than liability

One important distinction is often overlooked in discussions about AI governance: legal liability, operational accountability and technical traceability are closely related, but they are not the same thing.

Legal liability answers the question: Who is legally responsible when something goes wrong? This is determined through legislation, regulation and, ultimately, the courts.

Operational accountability answers: Who authorised the AI to act, and who is responsible for overseeing its use? Organisations remain accountable for the AI systems they deploy, including ensuring they operate within appropriate policies, permissions and governance frameworks.

Technical traceability answers: Can we prove what happened? It provides an auditable record of who authorised an AI system, what actions it performed, what information it relied upon and when those actions occurred. Without traceability, it becomes difficult for regulators, auditors or courts to establish the facts.

Each of these serves a different purpose, but together they form the foundation of trustworthy AI governance.

Digital identities, cryptographic credentials, secure authentication, audit trails and authoritative registers each address a different part of the trust equation. Together, they enable governments to verify identities, establish authority, record permissions and provide the evidence needed to support accountability and regulatory oversight.

Rather than replacing one another, these capabilities work together to create a trusted ecosystem for AI-enabled government services.

Different countries are exploring different approaches

While no single model has emerged, several initiatives demonstrate how governments and industry are beginning to address different aspects of AI governance.

One of the most ambitious proposals comes from Estonia, which has announced plans to develop digital identities for AI agents. The objective is not to grant AI legal personhood, but to enable autonomous systems to authenticate themselves, operate within clearly defined permissions and remain linked to the individual or organisation responsible for their actions.

In Canada, the Government of Canada AI Use Register takes a different approach. Rather than registering AI agents themselves, it provides public transparency into how AI systems are being used across government. It enables citizens to understand where AI is being deployed, what it is used for and which organisations are responsible for its implementation.

Outside government, a coalition of actors, musicians and creators has established a Human Consent Registry that allows individuals to record whether AI systems have permission to use their voice, likeness or identity. Although its purpose differs significantly from government registries, it reflects the same underlying principle: creating an authoritative record of rights, permissions and accountability.

These initiatives also sit alongside broader regulatory developments such as the European Union's AI Act, which establishes comprehensive obligations around transparency, governance, risk management and accountability for AI systems.

Taken together, these examples demonstrate that governments are not simply regulating AI. They are beginning to build the governance mechanisms needed to support it.

Why registries should be part of the conversation

Whenever governments need to establish who can act, under what authority, and with what legal effect, they turn to trusted registries. 

Corporate registries establish who is authorised to represent a company. Land registries record legal ownership. Intellectual property registers establish rights. Beneficial ownership registers improve transparency and accountability.

AI does not fundamentally change these principles. It introduces a new actor into systems that already depend on trusted records of identity, authority and legal responsibility.

If AI agents are going to submit statutory filings, interact with government services or act on behalf of businesses, governments will need infrastructure that can establish exactly the same things they already establish for people and organisations: identity, authority, permissions and accountability.

5 Key Questions Registry Leaders Should Ask About AI Governance

Whether AI registers become a reality in five years or fifteen, the conversation has already begun.

Registry authorities do not need all the answers today.

But they should be asking the right questions.

• What role should our registry play in future AI governance?
• Which existing registry capabilities could evolve to support AI identities, delegated authority or accountability?
• How can our existing platforms integrate with digital identity, authentication and interoperability frameworks?
• Could our legislative framework support trusted records of AI permissions and authority?
• Are we actively participating in the policy discussions that will shape AI governance in our jurisdiction?

The jurisdictions that begin exploring these questions today will be better positioned as AI regulation continues to mature.

At Foster Moore, we believe the future of AI governance is not simply about technology or legislation.

It is about building trusted, authoritative infrastructure that enables governments to regulate with confidence, businesses to innovate responsibly and citizens to trust the systems that increasingly shape their lives.

If these are conversations your organisation is beginning to have, we would welcome the opportunity to share perspectives, discuss emerging approaches from around the world and explore what trusted AI governance could look like for your jurisdiction.

 AI changes who performs the work. It does not change who is responsible for it. 

Ultimately, AI may change how work is performed, but it does not change where accountability rests. Organisations cannot outsource legal responsibility simply by delegating tasks to AI. Responsibility will continue to lie with the individuals and entities that authorise AI to act on their behalf.

The challenge for governments is ensuring they have trusted mechanisms to establish who granted that authority, under what conditions it was exercised and how those actions can be independently verified. That is precisely the kind of trust infrastructure registries have been built to provide.