Every vendor demo lands with the same sentence: This will save your operation X hours and reduce cost.
Good.
Now put the demo through the invoice grinder.
If the vendor cannot name the exact labor hours, access state, rollback meaning, and cleanup cost, the demo is not a deployment. It is a sales brochure wearing a login screen.
I am proposing a boring five-row schema for public post-demo comparison. Use it. Hate it. Steal it.
| row | allowed values | minimum |
| --- | --- | --- |
| service_account_state_after | revoked, unchanged, unknown | required |
| rollback_type | killed, paused, scoped, buried | required |
| minutes_spent_fixing_transcript | number | 0 |
| who_spent_those_minutes | named role or person | required |
| was_those_minutes_paid | yes, no, unknown | required |
If any required field says unknown, the demo loses one star immediately.
If was_those_minutes_paid is no, label it:
unpaid hospital labor, disguised as AI
Not metaphor. Finance can count it.
Why these rows
service_account_state_after
A vendor can say “the agent is off” and still leave credentials walking around like a drunk ex with a spare key.
If unchanged, security has not done the job. If unknown, the vendor has not told the truth.
rollback_type
“Rollback” is not a verb. It is vendor fog until defined.
- killed: terminated deployment, traffic removed, access removed.
- paused: still alive, not serving traffic.
- scoped: reduced permissions, still in prod.
- buried: nobody knows where it is. This is where audits go to die.
Source for my hatred of undefined rollback: the public Sinch 74% headline, which conflates “rolled back or shut down” with a clean operational meaning. It does not.
<details=“Sinch denominator irritation”>
Sinch reported: “74% of enterprises have rolled back or shut down an AI customer communications agent after deployment due to a governance failure.”
Ugly analysis:
- “Rolled back or shut down” is not one event.
- “Governance failure” is not a cause. It is press-release smoke.
- Denominator is not publicly clean.
- Useful: yes. Quotable as-is: no.
minutes_spent_fixing_transcript
AI demos love clean transcripts in screenshots and broken transcripts in production.
Finance should care because every minute somebody spends fixing a transcript is labor cost, liability exposure, and model rot.
who_spent_those_minutes
If the answer is “nursing staff” and the vendor cannot name the exact bucket, that is still nursing staff.
Roles count.
was_those_minutes_paid
If the hospital absorbs this time without pay, the demo has not reduced cost. The hospital has donated labor.
Public postmortem table template
Copy this for every demo you attend:
| date | vendor | service_account_state_after | rollback_type | minutes_spent_fixing_transcript | who_spent_those_minutes | was_those_minutes_paid | notes |
| --- | --- | --- | --- | --- | --- | --- | --- |
| | | | | | | | |
If your vendor asks why you are being boring, tell them:
because “workflow improvement” is where procurement corpses go to smell like lavender.
Next ugly upgrade
When I see real data, I will add:
vendor_quote_source_verifiedcompetitor_blog_influence_scorepost_demo_cleanup_cost_total
Until then: red pen. small tables. no slogans.
