Manage an open-source agent fleet with five records per agent

The question changes once the fleet is running

An agent fleet is several agents run as one unit, under one policy, with one record of what each of them did. Before launch the question is whether an agent works. After launch it is who may do what, and what they actually did. That second question needs an owner, a file, and a trail, and none of the three arrive on their own.

Fleet management is the work of keeping those records current. A team that cannot answer three questions about any agent in its fleet is not managing that agent. It cannot say which agent spent the money, which person approved the write, or what changed this month.

Five records per agent

Every agent needs five records. Write them down for the fleet you have now, even when it is three agents and one of them only drafts replies.

• Owner. A person's name, plus a second name for the weeks they are away. Keep it beside the agent's definition in the repo, in CODEOWNERS or a line in the team register. A change of owner then goes through the same review as a change of prompt.
• Permissions. What the agent may reach, tool by tool, and whether a call runs, waits for a person, or stops. In a project that keeps its definitions in a repo, the permission keys sit in the agent's block in kortix.yaml. They are text, so a widening appears as a diff.
• Approvals. Who said yes, to which action, and when. A connector rule set to Ask holds the action until a person approves it. An approval covers one exact action, so a changed recipient or a changed amount needs a new one.
• Runs. What the agent did and what it handed back. A run that ends with nothing a person can open deserves a look, whether or not it reported an error.
• Cost. What the agent spent, in the unit your bill uses. A shared credit pool gives one figure for the account, so the per-agent figure comes from matching runs to the bill.

Four of the five are files or rows that a person can read without opening a product. The fifth is a calculation, done by hand, from two records that live in different places.

Where each record lives

Most consoles show permissions, approvals and runs while the account is open. The question for a fleet owner is what stays readable when a person leaves, when the tool is replaced, or when a bill is disputed.

A fleet that keeps its definitions in a repo answers part of that by construction. The prompts, the permission keys, the skills and the memory are files in one git repo. The history of every change is a git log a reviewer can read line by line. The approvals and the runs arrive as records from the runtime that executed them. Cost is a join between those runs and the invoice, done by a person, on a schedule.

That division matters more than the console. A record you own can be read after you stop paying for the product that produced it, while a record that lives only inside a product leaves when the subscription does.

What a console does well, and what it keeps

LangSmith Fleet traces every run in LangSmith. It holds sensitive steps for approval in a central agent inbox, and admins decide which tools are available (langchain.com). For the run you are debugging this afternoon, that is the right surface to be on. FleetQ goes further on spend, with a per-tier risk policy of auto, ask or reject and a credit ledger with auto-pause on overspend (github.com).

Both products produce a real record. That record lives in their database, in their shape. Export it and you get rows about runs. The definitions that produced those runs, and the history of who widened them, stay in the builder where the fleet was set up.

On a repo-backed fleet that history is the diff. A permission widened in March is one line in a change request, with a name and a date attached and a reviewer who merged it. That is the part a console cannot give you, because the record it keeps is its own.

Four ways a fleet drifts

An agent that was never re-permissioned still reaches everything it was given on day one. The tools were set wide to get the first success, and six months later no review has touched that file.

The agent whose owner left keeps running. The permissions still resolve, the runs still finish, and nobody reads the output. The cheap fix is the owner record: a name and a backup, checked with the roster every month.

An approval can turn into standing consent. A person approves one send, and the next send, to a different recipient, goes out on the strength of it. Where an approval covers one exact action, the second send waits for its own person, which is the behavior a review should expect.

Cost shows up on the monthly bill as one figure for the account, with no agent attached. By the time finance asks which agent spent it, the trail is a month old and the person who would know has changed team.

The review that keeps the ledger true

The weekly review takes fifteen minutes. It looks at four lists: agents added or changed since last week, permission diffs, approvals granted, and runs that failed or ended with nothing to open.

The monthly review takes an hour. Compare the owner list against the team roster. Find agents with no runs in thirty days and retire them. Reconcile spend per agent against the bill. Reopen the definition diff for every agent whose permissions changed, and read what the change was for.

A fleet can run for a year on that cadence, because each item produces a record rather than a conversation, and that record is what a new owner reads after the next reorganization.

Start with the row

Pick one agent and write its five records into the repo this week. Let the second agent inherit the shape. A management model that fits in a table, with a file behind each row, is easier to keep true than a console nobody logs into between incidents.