Business automation lessons from the aviation industry

Since the 1980s, automating various flight management operations has contributed to a profound improvement in air transport safety and effectiveness. But a related human issue — automation dependency — has emerged as a significant challenge to further improvements in safety levels. Automation can contribute to diminishing manual flying skills and increasing complacency, as pilots avail themselves of automatic flight management and navigation systems to aid much of their decision-making.

In some cases, pilots don't fully understand the automatic processes controlling their sophisticated aircraft. The ironic enquiry "What's it doing now?" is sometimes heard in the cockpit, as pilots struggle to figure out the actions of the "automatics", as these systems are referred to on the flight deck.

Crucial as this is in the cockpit, automation dependency is equally problematic in many businesses today, whenever there is a disconnect between what managers think is going on and what is actually happening. The automation in question is not just technological, but also pervades the processes, algorithms, and reporting on which managers rely to inform their decision-making.

Aviation is addressing this phenomenon as a major problem and is seeking solutions. We suggest that many companies should do the same before their "business automatics" put them at risk of losing control.

Automation's effect on aviation

The first autopilot dates from 1914, when it enabled the aircraft to fly straight and level on a compass course without the pilot's attention. Since then, the complexity of automation in the cockpit has grown in line with increased functionality and regulation. It has undoubtedly contributed to a tremendous increase in safety. But whereas in 1914 the pilot remained in total control (thankfully, as the autopilot of that era was unreliable and controlled only one aspect of aircraft performance), over time automatics have delivered much more functionality in such a reliable way that pilots can totally rely on such systems to execute various complex tasks. However, there are downsides to this. The UK's Civil Aviation Authority warns that the introduction of more automation in the cockpit could lead to an increased risk of "loss of control" events:

• Automation has significantly changed the role of the pilot. With increased automation, pilots get less practice at flying the aircraft in all situations, so they are less able to deal with the loss-of-control events when they occur. They lose the "feel" for their aircraft, so they are less able even to recognise the subtle changes that precede a loss-of-control situation.
• Trainers can struggle to replicate all "upset" situations. Automation copes with the vast majority of situations that are possible to predict. But as it is very difficult to predict those situations not already managed by the automatics, it is also much harder to simulate these conditions and so train the pilots to deal with them.
• Automation makes real leadership in the cockpit even more crucial. The aviation industry has long recognised that the "authority gradient" on the flight deck needs to be appropriate, meaning that whilst responsibilities are clear, junior staff are encouraged to say what they think, even in the presence of an experienced captain, and to monitor each other's actions. Who is willing to speak up and ask the question, "What's it doing now?"
• Automation can be a distraction. Pilots can spend too long choosing from the menu of flight modes and forget about flying the aircraft. They may also spend too much time trying to understand the cause of an alarm, which sounds because a condition has been met (eg, stall), instead of taking the necessary action (eg, push the control wheel).

The aviation industry is addressing these issues by making changes to flight-deck training programmes, including a refocus on the continuing maintenance of manual handling skills, as a vital recourse when the automatics fail to work correctly. The sector is also addressing the requirement for deeper knowledge of the automatics themselves, and what exactly they are doing in any operational mode.

The effect of automation in businesses

Similar conditions can exist in companies if managers no longer understand in detail how their business processes function, how to recognise an emerging issue, or how to recover from a loss-of-control situation. Why?

• Managers assume that more automation must be the right thing to do. Employing fewer minimum-wage workers must be a good thing, right? Managers must properly evaluate the inherent risk — that is, losing the control and transparency they would have in a simpler system. For example, a sophisticated staff rostering system that uses its own algorithms could result in an airline not having enough pilots or crew in the right place at the right time.
• Managers often don't get involved in business process automation. They leave it to the project team and IT specialists, who set the parameters, prepare the workflows, and draw the colourful diagrams. But this abdication of responsibility means that the more efficient the automation is from an IT viewpoint, the less able the managers are to understand how the processes really work.
• De-layering has removed managers' go-to support staff. These used to be the people who monitored the information and systems and could go straight to the manager with any concerns. Increased automation has reduced operational and back-office staff numbers, and made those who remain "empowered". However, some managers now have to try to stay in control of more complicated processes despite reduced support, and without ever getting the time to understand how the processes function.
• Companies often neglect to identify complex but "routine" processes on their risk register. Not enough scenario or "what if" planning takes place, so managers remain unaware of the inherent risks until a problem occurs.
• There is too much information to assimilate. All the information generated by the automated business processes will be stored, transaction by transaction, somewhere — but how can it be disseminated in a form usable by managers? This may not have been specified.
• Humans are not programmed to monitor. The project team may have developed a reporting dashboard with colourful dials, gauges, and buttons, but do these really show what's going on — and what's about to happen? Choosing the information to include in reports means choosing what will be left out. Does the person making this choice really understand? Do the people monitoring the numbers know what the real questions are to ask?
• Automated processes rapidly become unfit for purpose. Businesses change quickly with mergers, acquisitions, divestments, downsizing, refocusing, and changed strategies. Going back to IT to respecify requests (eg, to make changes to cope with changing business needs, or simply to correct errors that have appeared) takes far too long — the business can't wait. Often, the original project team is now on some other critical activity, or they've left. So, staff will have to "just do it in Excel", or manually, with all the consequential control risks that entails. Even here, due to the inevitable time pressures, they'll just correct the minimum — top-down stuff of the "it'll do" variety. Management reports will no longer be able to capture the true story.
• People are afraid to question managers. The authority gradient is often far too steep, and this angle is actually increased by the removal of managers' support staff.

Recommendations for business managers

A famous axiom in the aviation industry is "a superior pilot uses his or her judgement to avoid situations that would require his or her superior skills." We recommend that business managers take this onboard by:

• Including all operational aspects of complex processes in risk reviews.
• Undertaking regular, formal "what if" audits of automated processes. For example, when the circumstances on which the model is based change unexpectedly, can the system cope with a different set of data or algorithms that reflect this change?
• In new process automation projects, including "loss of control" as a specific risk to be addressed, challenging whether more automation is really cost-effective, and ensuring that all data is easily accessed in open databases.
• Encouraging staff to open up and say what they think.
• Encouraging more simplicity in business processes.
• Ensuring they have real hands-on experience of their business automatics.

Peter Gillespie (info@meaningful-metrics.co.uk) is founder of Meaningful Metrics, based in Farnham, Surrey, UK, which provides management reporting services to medium-size organisations. He worked in manufacturing and services in several countries for 30 years. Darragh Owens (darraghowens@yahoo.com) is a professional pilot instructor based in Dublin, Ireland. He is also a writer and columnist on aviation training matters.To comment on this article or to suggest an idea for another article, contact Samantha White, an FM magazine senior editor, at Samantha.White@aicpa-cima.com.