Disclosure: I build Auditant, the last tool on this list. The other three are good, I use two of them, and the most useful thing I can tell you is that they are not competitors — they answer different questions for different people.
The question that decides which you need: who is going to ask you what your agent did?
- If the answer is "an engineer, when something breaks" — you need observability. LangSmith, Langfuse or Phoenix.
- If the answer is "a regulator, an auditor, or an enterprise customer's security review" — you need an audit trail. That's a different artefact, and none of the three observability tools was built to be one.
- If both — most teams shipping agents that touch money, healthcare, hiring or customers — you need one of each. They read the same spans.
What "audit trail" means, precisely
An observability trace answers "what did the model see and say, and how long did it take?" It is built to be cheap to write, quick to read, and short-lived, because traces are big and most are never looked at.
An audit trail answers "prove to me, without trusting you, that this sequence of actions happened in this order and nobody changed the record afterwards." It has four properties a trace doesn't:
- Retention matching the obligation. EU AI Act Article 19 says at least six months for high-risk systems; financial books-and-records rules run to years. Terraform Cloud keeps audit trails 14 days; GitHub keeps git audit events 7. The shortest provider retention is roughly 78× shorter than the shortest compliance obligation.
- Tamper evidence. Each entry carries a hash of the previous one. Edit any row and every link after it breaks.
- A named human on the approval. Not the service account that called the API — the person who authorised the action, taken from the authenticated session.
- Offline verifiability. An auditor can check the record on their own laptop, with public keys, without an account with you or the vendor.
Now the four tools against that.
LangSmith
For: teams building on LangChain/LangGraph who want first-party tracing, evals and prompt management in one place.
Deep LangChain integration, good eval tooling, a polished UI. It's an observability and evaluation platform, by LangChain, for LangChain users (it works with other frameworks, but the gravity is obvious). Traces are editable in the sense that anything in a mutable database is; retention is plan-dependent and measured in days to weeks on lower tiers. No tamper evidence, no offline verifier — it wasn't built for that and doesn't claim to be.
Use it for: debugging chains, running evals, iterating on prompts.
Langfuse
For: teams that want open-source observability they can self-host.
Open source (MIT), self-hostable, framework-agnostic, with a good OpenTelemetry story. Traces, sessions, scores, prompt management, cost tracking. Because you can self-host it you control retention — which solves property 1 above if you're disciplined about it — but a Postgres table you control is still a Postgres table you can edit, so properties 2–4 are not there.
Use it for: the same as LangSmith, when you'd rather own the deployment and avoid framework lock-in.
Arize Phoenix
For: ML/AI teams who want OpenInference-native tracing and evaluation, especially for RAG.
Open source, built on OpenTelemetry/OpenInference, strong on retrieval evaluation and embeddings visualisation. Same category as the two above with a research-flavoured UI; same absence of audit properties, for the same reason — it's a debugging tool.
Use it for: RAG quality, eval datasets, tracing across any framework via OTel.
Auditant
For: whoever has to answer a regulator, an auditor, or a security questionnaire about what an AI agent did.
This is the one I build. It records every agent action — model calls, tool calls, approvals, refusals — into a hash-chained log; checks each action against policy before it executes (above ₹X, a named person signs first); seals the chain head on a schedule and gets it countersigned by an independent RFC 3161 timestamp authority; and exports an evidence.json that anyone can verify offline with a public verifier, no account required. Retention is "as long as the obligation" and pricing is a flat band by agent count, never per event — metering events would pay you to record less.
It reads the same OpenTelemetry / OpenInference spans the three tools above emit, so if you already trace, you've done the instrumentation. It sits downstream of them, not opposite.
What it doesn't do: debugging. It holds fingerprints of payloads by default, not full prompts and completions, and its UI is a ledger, not a trace viewer. If the question is "why did the model say that?", you want one of the three above.
We run on it ourselves. Every ad-account change made by Admiral's agents — our paid-ads product — is recorded on the chain admiral/prod, and the live state of that chain (actions recorded, seals countersigned, sealed-through sequence) is on auditant.co/trust, read at request time, not written into the page. A customer whose ads Admiral runs can verify the change log offline without trusting either product.
The grid
| | LangSmith | Langfuse | Phoenix | Auditant | |---|---|---|---|---| | Built for | Debugging + evals | Debugging + evals (self-host) | Debugging + RAG evals | Compliance evidence | | Reader | Engineer | Engineer | Engineer / ML | Auditor, regulator, security reviewer | | Retention | Plan-dependent | You decide (self-host) | You decide | Matches the obligation, years | | Tamper-evident | No | No | No | Hash chain + independent timestamps | | Named human on approvals | Service account | Service account | Service account | Authenticated person | | Stops an action before it runs | No | No | No | Yes — policy runs pre-execution | | Verifiable offline by an outsider | No | No | No | Yes, public verifier | | Full payloads | Yes | Yes | Yes | Fingerprints by default | | Reads OTel spans | Yes | Yes | Yes | Yes | | Open source | No | Yes | Yes | Verifier and SDK yes; control plane no |
What to actually do
- If you have no regulator, no auditor, and no enterprise customer sending you a security questionnaire: pick one of the first three, and stop reading. You probably don't need an audit trail yet.
- If you ship agents that move money, approve applications, touch health or hiring data, or sell to enterprises in 2026: you need both. Keep the observability tool. Add a system of record that reads the same spans.
- Either way: check what your provider planes retain today. Terraform Cloud, GitHub, your LLM gateway. Write the numbers down. The gap between those and your shortest obligation is your exposure.
More on the category at auditant.co/ai-governance and the retention gap at auditant.co/llm-observability. I write about building software at 4ugusta.dev.