Trust Is Not a Number

The exercise

Open the marketing page of any AI platform you are considering. Find every sentence containing secure, private, trusted, safe, protected, or verified.

For each one, write down what it would take to falsify the claim.

Most of the time you cannot, because the claim does not say enough to be wrong.

• "Enterprise-grade security." Against whom?
• "Your data is private." Private from competitors, the vendor's employees, the vendor's cloud provider, or a subpoena?
• "Verified AI." Verified by whom, asserting what?

A system that is secure and robbed

Consider a system with these properties: every byte encrypted in transit and at rest, the model runs inside a hardware enclave so even the cloud provider cannot read its memory, the vendor holds a stack of compliance certifications.

By any reasonable measure, well protected.

Now an attacker puts a sentence on a web page. Your agent reads the page. It follows the sentence. It transfers money to the attacker.

Every encryption guarantee held perfectly. The enclave was never breached. The certifications were all genuine.

And the money is gone.

What failed was not confidentiality. It was authorization — the agent did something it should not have been permitted to do. Confidentiality mechanisms have nothing whatsoever to say about it.

This is not hypothetical. It is the most common way real agent systems are compromised.

Trust is a vector, not a scalar. A system fails at its weakest component, not its average.

Seven questions

Trust in agentic systems decomposes into seven distinct properties. Each answers a different question.

| Property | The question it answers | | --- | --- | | Confidentiality | Who can see the data? | | Integrity | Did the computation happen correctly? | | Authenticity / Provenance | Did this really come from where it claims? | | Identity | Is this the agent I think it is? | | Authorization | Was it allowed to do that? | | Accountability | Who is answerable when it goes wrong? | | Reputation | Should I have trusted it in the first place? |

These are not variations on a theme. They are genuinely different questions, and a mechanism that answers one usually says nothing about the others.

Encryption answers the first. It is silent on the other six.

Key takeaways

• Trust is a vector. Seven properties, different mechanisms, different attacks.
• Systems fail at the weakest property, not the average.
• The most common agent compromise is an authorization failure in a system with excellent confidentiality.
• A claim you cannot falsify is not a claim. Ask what would have to be true for it to be wrong.