Before an aircraft part ships, someone signs for it. Before an automated job runs against two hundred servers, increasingly nobody does — and that gap is where the questions in this piece come from. The teams we hear from are not short of tooling; they are short of answers about scope, safety and proof. So here are the questions worth settling before the next script goes live, answered the way we would want them answered: precisely, with the reasoning visible. The reference throughout is AutomationOps as documented in the Atlas’s AutomationOps catalogue entry — a proposed discipline for governing operational automation as if it were a production service, from selection through retirement.
AutomationOps Questions Worth Settling First
Order matters. Scope questions come before tool questions, tool questions before guard questions, guard questions before measurement. Skipping ahead is how estates end up with a dashboard full of jobs nobody can vouch for.
What exactly falls inside this discipline’s scope?
Run-time actions: the things software does to other software while systems are live. Restarts after a crash, temp-file clean-ups, failovers during an incident, key and certificate rotations, configuration pushes across a fleet. Things that look adjacent but sit outside: infrastructure declared as code belongs to InfraOps territory; multi-step workflow engines are a separate layer; commands typed into chat channels are ChatOps; network device changes are their own field; and measuring how much repetitive toil the team carries belongs to SRE practice. The boundaries matter because the controls differ. What this discipline does accept from its neighbours is candidates — SRE toil logs and incident reviews feed it the work worth governing — and it hands back controlled, registered jobs they can rely on.
Why does the term need careful handling?
Because the words have been used for other things. Robocorp described a methodology under this name, material that has since been archived (2023); UiPath ships a product of the same name that reached general availability in April 2021 for orchestrating RPA work. The Atlas’s meaning is different again: not a vendor category but an operating discipline for run-time automation of any kind. When you write an internal proposal, say early which sense you mean and cite the definition you are working from. Ambiguity here is not pedantry — it decides whether your budget line covers licences or controls.
Which tools make up a typical estate?
Most teams already own the pieces. Configuration management and ad-hoc execution live in tools such as Ansible and Puppet (with the OpenVox fork carrying the community banner). Runbook platforms — Rundeck, StackStorm — put buttons and audit trails in front of the same actions. Event-Driven Ansible reacts to signals automatically. The cloud equivalents are AWS SSM Automation and Azure Automation. Quality sits a layer down: ShellCheck for shell scripts, ansible-lint for playbooks, Molecule for testing roles. None of these is sufficient alone. A runner plus a register plus guard rails plus tests is what turns a toolbox into a governed system — and the register is the piece teams skip first, because it produces no output until the day it saves you.
How do you decide what may run untended?
By tier. The Atlas proposal sorts every job into four levels. Tier 0 is read-only — reporting, checks, dashboards — and needs little beyond an owner. Tier 1 makes a reversible change to a single target. Tier 2 makes reversible changes across many targets and therefore earns dry runs, batched execution and health checks between waves. Tier 3 is irreversible or data-destroying, and its controls are the strictest: a second person must approve, deletion should be soft where possible, and rollout proceeds in stages. Tier 3 lines up with dual-authorisation expectations in NIST’s AC-3(2) and CM-5(4). A maintenance-planning analogy holds: routine inspection, minor repair, fleet-wide service bulletin, then engine teardown — each step demands more paperwork because it can do more harm. The tier is not a judgement about the script’s quality; it is a statement about what the script is capable of breaking, which is why the same tidy job can sit at tier 1 in a lab and tier 3 against customer data.
What makes a repeat genuinely safe?
Two properties. First, idempotency: the job expresses a desired state — “ensure the service is running” — rather than issuing a bare action like “start”, so running it twice changes nothing the second time. Molecule’s default test sequence even includes a dedicated idempotence step for exactly this reason. Second, wherever an action cannot repeat safely — a create, a charge, an outbound message — the caller supplies a request ID and the system records it atomically, so a retry is recognised and refused rather than doubled. Google’s internal repair tooling is the standard anecdote: fixes could be re-applied on a fifteen-minute cycle without fear, because neither property was left to luck. Ask for both properties by name when you review any proposed job; their absence is the earliest visible warning.
What should be measured once jobs go live?
A short list, kept honest:
- Net hours returned. Runs per month times manual minutes, minus residual human minutes, divided by sixty, minus maintenance hours.
- Run success rate — with failures triaged, not just counted.
- Human fallback rate. How often a person had to take over.
- Induced-incident rate. Outages the automation itself contributed to.
- Guard trip rate. How often a dry run, batch cap or budget stopped something — trips are the system working.
- Time to stop. Measured in drills, not estimated in slides.
- Ownership coverage and the count of unregistered jobs still hiding in personal directories.
The worked example worth memorising: at forty runs a month, a job returns about 7.83 net hours and covers its build cost in roughly 5.1 months while succeeding 95 percent of the time. Numbers like that survive contact with a finance reviewer. Two cautions apply when you collect your own. Report hours returned after maintenance is subtracted, or the figure flatters you within a quarter. And treat a sudden improvement in success rate with suspicion if the human fallback rate fell at the same time — it may mean failures are being absorbed quietly by people rather than fixed.
Where should a small operations team begin?
With an inventory and an honest ranking. List everything that runs unattended, register the twenty riskiest with named owners and tiers, and do nothing else until that list is true. Month two: consolidate execution through one runner, separate identities for each job, and hold the first kill-switch drill — on-call engineers should be able to halt any job without seeking permission first, and the drill proves it. Month three: promote jobs whose evidence supports promotion, and push ownership coverage above ninety percent. The career-side reading on growing an engineering career pairs with this sequence, and the piece on cloud operations training and certifications helps you route the team’s study time. For an adjacent discipline with the same governed-system mindset, see the site’s take on modern RobotOps strategy — different actuators, identical bookkeeping.
AutomationOps, Answered and Ready to Use
Scope settled, tools named, tiers assigned, repeats made safe, numbers agreed, first twenty registered. Any team that can answer these seven questions in writing is further along than most — not because the answers are complicated, but because asking them in order, before the next job ships, is still the exception. Be the exception; the audit trail will read like a pre-flight sign-off, and it will hold up exactly when it matters.

