automation risks

The Pitfalls of Automation: Unexpected Consequences and Responses in Software Systems

Key Takeaways * Automation can behave in counterintuitive ways during software incidents — sometimes impeding resolution or complicating human intervention. * Strict separation of tasks into “automation” vs. “human” can lead to designs that make incidents harder to resolve. * Overuse of automation can erode human knowledge, skills, and situational awareness. * Joint Cognitive Systems