Accountability and Reputation

Accountability — who is answerable?

The agent spent $8,000. Everyone agrees that happened. Nobody agrees whose fault it is.

Accountability means deviations produce undeniable evidence attributing the action to a responsible party. Two components:

Non-repudiation — the actor cannot later deny having acted. This requires cryptographic binding, not logging. A log your operator can edit is not evidence; it is a record of what your operator wanted you to see.

Attribution — there is an identifiable party who answers. Partly technical, substantially legal, and the legal side is genuinely unresolved. When an autonomous system causes loss, the candidates include the model provider, the framework developer, the operator, and the human principal.

Accountability is the property most often absent from otherwise sophisticated systems — because it provides no benefit until something goes wrong, at which point it provides all of its benefit at once. That is a hard thing to fund. It is exactly why it should be a design requirement, not an afterthought.

Reputation — should I have trusted it?

The forward-looking property.

Identity tells you who. Accountability tells you who to blame afterwards. Reputation tries to tell you, in advance, whether to engage at all.

This matters once agents transact with strangers — your agent needs to hire another agent, chooses between several, and has no prior relationship with any.

Mechanisms include signed feedback from past counterparties, third-party attestations, staked bonds, and aggregation algorithms that resist manipulation. Every one is gameable, in ways well understood and cheap to execute.

Reputation is a lagging indicator built on identity and accountability. Without stable identity, ratings attach to nothing. Without accountability, there is no ground truth for ratings to reflect. Systems that build reputation without those foundations produce numbers that look meaningful and are not.

The table, filled in

| Property | What we want | Currently provided by | Status | | --- | --- | --- | --- | | Confidentiality | Email contents not exposed to model host or network | TLS in transit; nothing at the host | Partial | | Integrity | The advertised model actually ran | Nothing | Absent | | Authenticity | The confirmation came from our agent | Email domain authentication, weakly | Partial | | Identity | The booking service is who it claims | TLS certificate for the API | Partial | | Authorization | Spend capped, travel only, this trip only | The model's judgement | Absent | | Accountability | Evidence of what was done, on whose authority | Application logs, operator-editable | Absent | | Reputation | Confidence in the booking service | Brand recognition | Informal |

Three absent, three partial, one informal — and this is a system a reasonable team would consider shipped. Every gap arose because the property was never named as a requirement.

Case study: six out of seven

A company deploys an internal agent that reads their document repository and answers employee questions. Done carefully:

• Documents encrypted at rest
• TLS in transit
• Model on infrastructure they control
• Access requires SSO
• Every query logged
• Vendor certified against standard frameworks

Confidentiality strong. Identity strong. Authenticity fine. Accountability reasonable. Integrity adequate.

Six properties, well handled.

Then someone notices the agent will answer questions about documents the asking employee should not have access to.

The document repository has per-user permissions. The retrieval index does not. The agent authenticates the user perfectly — then queries the corpus with its own authority, which is total.

Authorization: absent.

The result is a genuinely well-engineered system that also functions as a comprehensive permissions-bypass tool for anyone who asks it nicely.

The lesson generalises: ask which property nobody owns. That is usually the one that fails.

Key takeaways

• Logs your operator can edit are not evidence. Accountability requires tamper-evidence and cryptographic binding.
• Reputation depends on identity and accountability. Without them it produces meaningless numbers.
• Our travel agent currently has three absent properties — and looks shipped.
• Ask which property nobody owns. That is usually the one that fails.