The agent never printed the key. Check for its digest too.

A digest is not the key, but it is not nothing

A SHA-256 digest of a long random token does not give the token back. For high-entropy, freshly generated secrets, a digest in a log is not the secret in a log, and treating every digest as a compromise would be wrong.

A digest can still let an outsider, or another agent, do things you may not intend:

• Confirm a guess. Anyone holding a candidate value can hash it and compare. For low-entropy secrets (short passwords, predictable formats, values reused from elsewhere) that turns a digest into an offline guessing target.
• Link records. The same digest in two systems says the same secret sits behind both, even to a reader who never sees the secret.
• Outlive the vault. History and receipts are often kept longer, and read more widely, than the secret store.

Whether this matters for you depends on the secret's entropy, who reads each output, and for how long. The simpler default: outputs that do not need a secret-derived value should not contain one, and a test should check that directly.

Where derived values end up

The secret usually has one home, a vault or secrets manager. Metadata about it has many:

1. Tool and API responses returned to the agent, including cache or lookup fields.
2. Activity feeds that record what changed, often as before/after values.
3. Audit logs, including older records written in a previous format.
4. Receipts and summaries produced for humans or other agents.

Two shapes are easy to miss. Nesting: the value sits four levels down, or is a dictionary key rather than a value. Legacy data: an older record may store metadata as a JSON document inside a string field. A checker that skips string fields can miss it.

The toy check

The script invents a fake secret (FAKE-DOCS-ONLY-not-a-real-key-0123456789), four made-up output surfaces, and a checker. It treats three representations as forbidden: the raw value, its full SHA-256 hex digest, and a 12-character prefix of that digest. Hex is matched without regard to case, so sha256:9E3A05... counts.

The checker walks every key and every string value. Substring matching on the outer string catches the legacy plant; when a string is JSON, the checker also parses it and reports the nested path:

def walk(node, path="$"):
    if isinstance(node, dict):
        for k, v in node.items():
            if isinstance(k, str):
                yield f"{path}.<key>", k
            yield from walk(v, f"{path}.{k}")
    elif isinstance(node, list):
        for i, v in enumerate(node):
            yield from walk(v, f"{path}[{i}]")
    elif isinstance(node, str):
        yield path, node
        s = node.strip()
        if s[:1] in "{[":
            try:
                inner = json.loads(s)
            except ValueError:
                return
            yield from walk(inner, f"{path}<json>")

A checker that never fails proves nothing, so the script first runs negative controls: it plants each forbidden representation at each of six placements across the four surfaces (nested values, a dictionary key, the legacy record's embedded JSON) and requires the checker to flag every plant. A raw-value-only check runs against the same plants. Then the clean outputs are checked.

$ python3 secret_metadata_check.py
...
  full check detected 18/18
  raw-only check missed 12/12 digest plants

clean outputs: 0 forbidden hits across 4 surfaces

RESULT: PASS (toy example; not evidence about any real system)

The exit code is 0. With --demo-leak, the short digest is planted in the legacy record before the clean check. The run reports it twice (outer string, decoded path) and exits 1. It needs Python 3 only: no network, accounts or real credentials.

The number that matters is the second one: the raw-value-only test missed every digest plant.

Adapting it

On a real system, replace the four builders with captures of your actual outputs, produced in a test environment with a throwaway secret generated for the test. Then:

• List every surface a secret-bearing operation writes to, not only the ones the agent reads.
• Include historical records in their stored shape, not just the current schema.
• Add each representation your code can produce: other hashes, keyed hashes, base64, truncations. The script covers three.
• Keep the negative controls. A new surface gets a placement, and the check must still fail on a plant.

A clean run says these outputs, for this input, did not contain these strings. It says nothing about what an attacker can do or what you forgot to capture, which is why the list of surfaces should be written down and reviewed.

The script is released under the MIT No Attribution license (MIT-0). All values, names and output formats in it are invented for this example.