How to Score a Workflow for AI-Agent Safety Before You Automate It

The question comes up in every planning meeting now: "can we just have an agent do this?" Someone points at the backlog — triaging support tickets, drafting PR descriptions, reconciling a vendor invoice, writing the weekly investor update — and nobody in the room can say, with any real confidence, which of those is safe to hand to an AI agent and which one ends with a customer getting a refund nobody approved. The debate usually resolves the same way: someone says "let's just try it and see," and the org ends up automating whatever an engineer happened to prototype over a weekend, not whatever the business could actually tolerate getting wrong.

That's a bad way to decide, because the workflows that are easiest to demo — clean prompt, obvious happy path — are not the workflows that are safe. A demo shows you the agent can do the task. It says nothing about what happens on the input that doesn't look like the other forty-nine, and agents don't fail the way a broken script fails. A script that hits a case it can't handle throws an exception. An agent that hits a case it can't handle writes a fluent, confident, wrong answer and ships it, because nothing in the architecture forces it to know the difference.

Why "just try it" doesn't hold up

The instinct is to run a pilot: give the agent ten real tickets or ten real PRs, grade the output, and expand if it looks good. That tells you the agent is competent at the median case. It tells you almost nothing about the tail, and the tail is where the cost lives — the one reply that states a fact about a customer's account that isn't true, the one commit that touches a file the reviewer didn't expect to be in scope, the one refund that's technically correct but violates a policy nobody wrote down because it never came up before.

Adding a human reviewer doesn't fix this by itself, either. Review only catches what the reviewer is actually positioned to catch. If the output is confident and well-formatted, someone skimming twenty of these a day will approve the wrong one exactly like they'd approve the right one — automation complacency is a well-documented failure mode wherever people supervise automated output, not a hypothetical. A review step has to be built to catch a specific class of failure, or it's a signature on a form.

Score the workflow before anyone writes a prompt

The variable that actually predicts whether a workflow is agent-safe isn't the job title attached to it, or how repetitive it looks, or how good the writer prompt gets after a few rounds of tuning. It's a small number of properties of the task itself:

• Reversibility — if the output is wrong, how expensive is it to undo? A draft nobody's read yet is trivially reversible. A reply already in a customer's inbox, a merged commit, a payment, is not.
• Blast radius — who or what does a bad output touch, and how far does it travel before a human sees it? Internal-only and single-record beats external-facing and batch.
• Verifiability — can correctness be checked by something other than a person's judgment call, before the output takes effect? A build either compiles or it doesn't. A tone judgment on a customer email isn't checkable that way unless you've built a specific test for the thing you actually care about — no invented facts, no dead links, no promise the contract doesn't cover.
• Ambiguity — does the task have one correct shape, or does "correct" depend on context the agent doesn't reliably have? Renaming a variable is unambiguous. Deciding whether a bug is a P1 depends on judgment the ticket text doesn't always contain.

Score each workflow across these axes, low score meaning high risk, and you get something better than a gut feeling: a ranked list. The tasks that land high on all four — reversible, narrow blast radius, machine-checkable, unambiguous — are worth fully delegating first. The ones that score low on reversibility or blast radius stay human-owned regardless of how well an agent performs in testing, because the rubric isn't measuring competence. It's measuring what a mistake costs.

The genuinely tedious part is what comes after the score: building the actual gate for whatever lands in the middle. That means writing down, in specific and boring detail, the failure modes for *that* workflow — not "the agent might hallucinate" but "the agent might cite a discount policy that doesn't exist" — and then building a deterministic check for each one that runs before the output reaches a customer or a system of record. That's schema validation, link checks, policy-lookup verification, a second model doing blind adversarial review against rules you've actually written down. Not a person eyeballing the output and deciding it looks fine.

When to skip the exercise

This is overkill for a lot of what people ask about. If a workflow is high-volume, low-blast-radius, and cheap to check — formatting a changelog, drafting an internal summary someone will read anyway, tagging incoming tickets by category — ship it and watch the error rate. Building a four-axis scoring model for a task where being wrong costs someone five minutes is process for its own sake, not risk management.

And no amount of scoring turns a genuinely irreversible, high-blast-radius task into a safe one. Anything that moves money, changes production infrastructure without a rollback path, or makes a legal or medical claim on the company's behalf belongs to a human today. The honest answer when someone asks "can the agent do this" is sometimes "not yet, and maybe not without a different architecture" — not a more elaborate gate. A rubric is for triage, not for talking yourself into delegating something you already know is too expensive to get wrong.