AI incident tools are taking over investigations that once gave on-call engineers regular practice with their systems. In an essay published on September 4, Sylvain Kalache argues that this will shorten most outages while making complex ones take longer to resolve. The essay reached the front page of Hacker News the next day, drawing 410 points and 341 comments.

Kalache leads AI Labs at Rootly, ran SRE at LinkedIn in 2012, and co-founded Holberton School. He describes AI responders that inspect alerts, develop hypotheses, query telemetry, connect problems to recent deployments, and sometimes apply the fix. When a tool handles that whole investigation, the engineer's role can shrink to approving its work.

His prediction concerns the incidents left over. Routine failures teach engineers how a system behaves under stress. If automation removes those encounters, the person carrying the pager may have less experience to draw on when the tool gets stuck. Mean time to resolution, or MTTR, could improve for most incidents even as resolution times rise sharply for the difficult ones.

An automation problem described in 1983

Lisanne Bainbridge described the same problem 43 years earlier in “Ironies of Automation”, a four-page paper published in Automatica in 1983. Automating routine work leaves people responsible for rare, abnormal situations while taking away the daily practice that prepares them to respond. Skills deteriorate without use, and the remaining job often involves prolonged monitoring, a task people tend to perform poorly.

The paper had been cited about 1,800 times by 2016. Its argument remains useful because each new layer of automation can create the same dependency: a system needs a capable human backup but provides fewer opportunities for that person to stay capable.

Kalache uses aviation as an example. Modern turbine engines have fewer than one in-flight shutdown per 100,000 engine flight hours. At that level of reliability, a pilot can spend an entire career flying without experiencing one. Recurrent simulator training provides practice that normal operations no longer supply. The FAA requires that training every six months, including engine-failure scenarios.

TransAsia Flight 235 shows the consequences of a failed response. On February 4, 2015, an ATR 72 lost its right engine 37 seconds after takeoff when a faulty autofeather unit shut it down. The crew then shut down the left engine, which was still running. The aircraft crashed into the Keelung River in Taipei 142 seconds after takeoff. Of the 58 people aboard, 43 died. Taiwan's Aviation Safety Council found that the crew had failed to follow the engine-failure identification procedure.

That accident illustrates the need to retain emergency-response skills. It doesn't establish how quickly engineers lose those skills under AI assistance.

The practice automation removes

The operational concern predates AI. Configuration management reduces the need to build a server by hand. Managed databases can reduce exposure to recovering a corrupted replica. Autoscaling can mean fewer encounters with capacity limits, while cloud load balancers can reduce occasions to inspect packet captures. These tools can still be worth deploying. Their benefits don't remove the need to practice the work that remains a human responsibility.

Backups offer a useful comparison. A backup that has never been restored is still a hypothesis about recoverability. A successful nightly job doesn't establish that a restore will work when needed. A missing key, an incorrect path, or a 14-hour transfer may become apparent only during a test or an emergency.

The same distinction applies to incident response. Watching an AI agent resolve 40 pages doesn't demonstrate that an engineer can resolve the 41st without it. Familiarity with successful automation and the ability to investigate a failure are different things.

AI may also shorten the time available to notice the loss of practice. Older automation often took over one control-room function at a time. An AI responder can take over the investigation loop in one deployment. The concern is that a skills gap that once developed over years could develop over a couple of quarters.

Rootly sells both automation and training

Kalache's employer has a commercial interest in the proposed response. Rootly sells AI incident tooling and launched Rootly Academy on March 24 at SREcon in Seattle. The academy uses incident simulations from Uptime Labs.

The essay therefore describes a problem that Rootly's automation could contribute to, while Rootly also sells training intended to address it. That relationship doesn't invalidate the argument. Brent Chapman, author of Building Internet Firewalls, made a similar case on his own blog in June without a product attached. It does give readers a reason to examine the recommendation for paid training separately from the underlying need for practice.

Much of the Hacker News discussion moved from on-call work to code generation. One commenter reported that their company's objectives and key results, or OKRs, demanded heavy AI use above all else. That is one person's account, but it describes a specific management risk: measuring adoption while leaving incident readiness untested. An improving MTTR dashboard can coexist with declining manual-response capability.

Keeping engineers ready without buying another tool

Keeping an AI responder doesn't require giving up independent investigation. Four practices can preserve opportunities to learn and expose gaps before a production emergency:

  • Record a hypothesis before reading the AI's assessment. A responder can write one sentence in the incident channel, then compare it with the tool's explanation. This keeps forming a theory from incomplete evidence part of the job.
  • Run a quarterly game day with AI tooling disabled. Break something in staging, page a human, and time the resolution. Track that result separately from production MTTR, which increasingly reflects the tool's contribution rather than unaided human performance.
  • Rotate incident command beyond the senior engineers. If the same two people lead every difficult incident, the team concentrates both experience and responsibility in people who may eventually leave. Other responders need practice leading investigations.
  • Require evidence behind the AI's conclusions. An explanation that deploy 4412 was rolled back because p99 latency rose during its rollout gives an engineer something to inspect and learn from. A status message that only reports the problem as fixed doesn't.

Ken Ashe published a similar list on September 5 and acknowledged that the evidence remains thin. The case rests on practitioner observations rather than studies measuring skill decay in AI-assisted on-call rotations. The tools haven't been in production long enough to establish the claimed long-term effect. Bainbridge's original evidence was also limited, though the argument has remained relevant for four decades.

A specific prediction follows from this concern: within two years, a large company's public postmortem will identify AI response tooling as a contributing factor in an outage lasting hours because the responders had never performed the manual procedure the agent failed to complete. The further prediction is that the remedy will include a training program, likely supplied by a vendor. Manual-response drills offer a way to test readiness before an outage tests it instead.