How to audit a system with an AI agent that never gets the keys

By the end of this setup, a team auditing a system it didn't build has a report in layers, from one page to a full write-up, where every claim points to a file. The reading is done by a coding agent that never touches a live system and never sees a credential.

The agent is good at the slow part: reading thousands of lines of metadata, tickets and wiki pages, and keeping track of what it found. It's also the riskiest reader you could hand a token to. Tickets and wiki pages are text anyone could have written, and an agent that holds a live credential can be steered by what it reads. So the work splits cleanly: people touch the systems, the agent touches files.

What you need before starting

• A read-only account on each system you're auditing. Ask the owner for read-only access on day one. If they can only give you admin, that's your first finding.
• The OS keychain, to hold the tokens.
• A git repo for the audit itself: plan, briefs, findings and report.
• A coding agent whose permissions you can restrict: no network, no platform command-line tools, and file access only inside the repo.

Write the plan and the tracker first

The plan splits the audit into workstreams: preparation, technical, risk, operations and the report. Each workstream is a list of steps, and the tracker says which steps are done, in progress or waiting on someone outside the team.

Add a glossary at the same time. An inherited system has its own names for things, and the agent uses whichever name it saw last. A glossary in the repo keeps the agent and the people on the same words.

Run every export yourself

Anything that talks to a live system is a person's job. Each export is a small script that a person runs, and it reads its token from the keychain at that moment:

#!/bin/sh
# Run by a person, never by the agent.
set -eu
token=$(security find-generic-password -s audit-tickets -w)
curl -fsS -H "Authorization: Bearer $token" \
  "https://tickets.example.com/api/search?updated-since=2026-01-01" \
  > "inputs/tickets-production-$(date +%F).json"

The token never sits in an env file or in the repo, so there's nothing for the agent to find.

Exports go into an inputs/ folder listed in .gitignore. Each file name says which system, which environment and which day it came from. Two environments of the same system look almost identical in an export, and a finding about the wrong one is worse than no finding.

Export metadata and configuration, not records. You need to know that a field holds email addresses, not the addresses themselves. Where records can't be avoided, write that down, because the inputs folder now holds personal data.

Then set the agent's permissions to match. Deny network access and the platform's command-line tools, and allow file access only inside the repo. The agent can still run summary scripts you give it over files already on disk. It just can't reach anything live.

One brief per area, before any findings

Each area of the audit gets a brief before the agent reads anything for it. The brief says which question the area answers, which inputs to read, what's out of scope, what a finding looks like and when the area is done:

# Brief: connections and logins

Question: which external systems can log in, and with what?
Read: inputs/metadata-production/connected-apps/, inputs/wiki/integrations.md
Out of scope: code quality, the data model
Finding: claim, evidence (file and line), confidence, severity
Done when: every connection in the metadata is listed, or marked unknown

Without a brief, the agent summarizes everything it can see. Findings come back without evidence, and the same issue turns up in three areas with three different severities. The brief is also what a reviewer checks the findings against.

Typical areas are a system overview, connections and logins, code quality and tests, a personal data map, a ticket review and the operations around the system.

Findings with evidence, and a register for what you inherit

Every finding has four parts: the claim, the evidence as a file and line, a confidence level and a severity. A finding with no evidence doesn't go in.

Some things aren't findings about the system's design. They're problems the next owner inherits: an expired certificate, a job that fails every night, an integration nobody can explain. They go in a separate register, so the report stays about decisions and the register becomes a work list.

Before anything reaches the report, check it:

• Drop every finding that has no file to point at.
• Check a sample by hand, and check more of the high-severity ones.
• Treat documentation and meeting notes as claims. Test them against tickets and metadata, and record where they disagree. The disagreement is often a finding of its own.
• Make sure every finding names its environment.

Layer the report

Different readers need different depths, so the report has three layers:

• One page: the situation, the main risks and the recommendation.
• Three pages: the options, with timing and commercial shape for each.
• An internal memo for your own team: what you would decide and why, kept apart from the version anyone else reads.

Every claim in every layer links to a finding, and every finding to a file. A reader who doubts a sentence on the one-pager can follow it down to the export it came from.

Hand back and clean up

When the audit closes, the inputs folder is the most complete copy of the system that exists outside the system itself. Delete it or hand it back to the owner, and say which in the report. Keep the registers. They're the part the team that takes over will use.