Skip to main content

Kill Switch, Shadow Mode and Examiner Reports

Bank supervisors ask four questions about AI agents that move money:

QuestionWhere Auditail answers it
What limits are placed on the agents?Policies: default-deny, conditions and limits over time
How are their actions approved and monitored?Human approval and the audit log
Can their decisions be tracked and audited?The hash-chained log and signed evidence
Can they be stopped?The kill switch, below

This page covers the kill switch, how to roll out a policy safely (shadow mode and backtests), and the signed examiner report that answers all four questions for a period.


Kill switch​

A freeze stops agents straight away. You can freeze:

  • every agent on the account ("scope": "all"),
  • one agent ("scope": "agent", "target": "payout-agent"),
  • one action ("scope": "action", "target": "create_payout").

Anyone on the account can freeze. Only an approver can lift a freeze. An agent, or your own monitoring with an agent key, can pull the brake on itself, but it can't release it.

From the dashboard: type a reason and click Freeze all agents. A red banner shows every active freeze; approvers see a Lift button on each.

From code or a runbook:

curl -X POST https://api.auditail.io/v1/freeze \
-H "Authorization: Bearer $AUDITAIL_API_KEY" -H "content-type: application/json" \
-d '{"scope": "agent", "target": "payout-agent", "reason": "Unusual payout volume, investigating"}'
# 201 { "id": "frz_…", "scope": "agent", "target": "payout-agent", "active": true, … }

curl -X POST https://api.auditail.io/v1/freeze/frz_…/lift -H "Authorization: Bearer $AUDITAIL_APPROVER_TOKEN"

While a freeze is active:

  • guard() and POST /v1/decide refuse matching actions with code frozen, even when the policy is in shadow mode. The hint tells the agent to stop and tell the user, because retrying won't help.
  • New approval requests for matching actions are refused (423), and pending ones can't be approved until the freeze is lifted. That way nobody approves a payout while an incident is open.
  • Freezing and lifting are both written to the audit log (freeze.activated, freeze.lifted) and sent to your webhook.

How quickly it applies. With serverPolicy: true, the server checks every action, so a freeze applies to the next call. If you only run rules locally, the SDK checks freeze status every 10 seconds (freezeCheckIntervalMs in JS, freeze_check_interval in Python; 0 turns the check off). A newly started process waits up to 1.5 seconds for the first check, so it respects a freeze that's already active.


Shadow mode: see what a policy would do before enforcing it​

Add "mode": "shadow" to a policy:

{ "mode": "shadow", "defaultAction": "deny", "actions": { "create_payout": { "limits": [ … ] } } }

In shadow mode the policy is evaluated in full and real usage is counted against its limits. Every action is still allowed, and the response tells you what would have happened:

const r = await auditail.guard({ action: 'create_payout', payload, agentId: 'payout-agent' });
r.approved; // true
r.shadow; // { wouldBe: 'require_approval', code: 'limit_exceeded', reason: '…' }

The dashboard marks these events Shadow: would block or Shadow: would need approval. Reports → Shadow report (or GET /v1/report/shadow) adds them up: how many actions would have been denied or sent to a human, and the total amount that would have been held. When the numbers look right, remove "mode" (or set "enforce") and save.

The kill switch still applies in shadow mode.

Backtest: replay recorded traffic against a policy​

POST /v1/policy/simulate replays the last 7 days of recorded actions, in order, against your current policy or a draft, and changes nothing:

curl -X POST https://api.auditail.io/v1/policy/simulate \
-H "Authorization: Bearer $AUDITAIL_API_KEY" -H "content-type: application/json" \
-d '{"policy": { "defaultAction": "deny", "actions": { … } }, "from": "2026-11-01T00:00:00Z"}'

The response lists how many actions the policy would allow, deny or send for approval, broken down by code and action. It also shows which actions were allowed at the time but would now be stopped, the value held, and up to 25 examples. In the dashboard: Reports → Backtest current policy.

What a backtest can't know, plainly:

  • Limits are counted from the start of the range, so usage from before the range isn't included.
  • An action the policy would have stopped is treated as not executed, so it doesn't count toward limits.
  • Provenance is only known for events recorded with a source; sources of individual fields aren't stored on events.
  • At most 20,000 events are replayed (truncated: true if there were more).

Examiner report​

GET /v1/report/examiner (default: the last 30 days; set ?from= and ?to= for another period) produces a document a compliance officer can hand to an examiner or auditor. In the dashboard: Reports → Examiner report, then Download signed report or Print.

It contains:

  • a plain-language answer to each of the four questions, with the figures behind it: limits in force, approvals by approver, median and 90th-percentile time to decision, approvals that overrode a limit or an untrusted-data flag, and every freeze with who pulled it, why and for how long;
  • the policy in force at the start of the period and every change during it, with who saved it;
  • the result of a full integrity check of the log, plus a signed checkpoint;
  • an Ed25519 signature by Auditail over the whole report.

Anyone can check the signature offline:

npx -p auditail-sdk auditail-verify auditail-examiner-report-key_xxx.json

or in code with verifyReport(report, { account: 'key_…' }) from auditail-sdk. Changing any figure in the file makes verification fail.

The report describes what Auditail recorded and enforced. It supports your audit evidence; it doesn't certify compliance with any regulation or framework.