An electronic failure of an electrical component within a power room caused an outage at the Dane County Regional Airport in a snap of cold weather on Jan. 21, 2019 in Madison, Wis. Although it did not create the national travel impact of the electrical outage at Atlanta’s Hartsfield-Jackson International Airport in Atlanta over a year ago (December 2017), it still affected airport operations within the terminal facility and disrupted flights.
Airport officials blamed the cold weather, but come on, it’s Wisconsin. They should know they need to design redundant systems to withstand cold weather.
I discussed the importance of resiliency in my last article. This issue keeps popping up at various airport facilities nationwide where the design spec should be that all facilities should have mission-critical (read: fault tolerant) infrastructure and redundant systems. If there is a failure, there should be an automated switchover to a redundant system or sub-system to ensure business continuity. Operations should not stop.
Another major airport that recently suffered power failure and subsequent flight cancellations was Bradley International Airport in Hartford, Conn. In November of last year, the outage there was more significant, and thousands of passengers were left in chaos until they could get airport operations back on track.
Also in that same month, Philadelphia International Airport experienced a power outage, stopping flights and crippling operations.
Earlier in August of 2018, Reagan National Airport experienced a power outage as well.
It seems like we are seeing more examples of poor designs within mission-critical airport facilities, which lack this type of design rule of thumb—true redundant systems.
These failures and outages can cost millions to tens of millions of dollars. In just the Hartsfield-Jackson Airport power failure, one airlines (Delta), calculated that it probably lost $25–$50 million from cancelling 1,400 flights.
So where are all the system design mistakes that we have yet to uncover before they cause significant interruptions and financial damage?
Should we perform more audits?
This is a critical question that all administrators of airports, pipelines, railroads, highways, and other layers of infrastructure should be asking as we see problems arising from a lack of good mission-critical infrastructure design. Before the failure occurs, we should evaluate how resilient a facility is, both in its power distribution as well as its broadband connectivity.
As business changes and threats evolve, this should be an annual exercise and internal review, not something done once a decade. Several questions need to be answered on an annual basis that include the following:
- Is our facility resilient? Can it withstand a natural disaster?
- Can it withstand a man-made disaster?
- Can it withstand acts of sabotage and physical intrusions?
- Can the facility withstand cyberattacks, electronic intrusions, and cyberwarfare
- Are we missing something that we need to incorporate into our planning and design based on new security threats? Drones? Other emerging threats, like electro-magnetic pulse (EMP)?
We could go on with a more detailed list, but you get the idea. This is something that should be incorporated into the annual review of all facilities dealing with transportation, healthcare and other critical components of commerce and layers of infrastructure.
The whole platform for commerce should be well-insulated from equipment and system failures. It should not be designed, nor managed, by those who have an urge to do things cheaply or those who do not understand the subtle design differences between business continuity and disaster recovery.
If you are involved in selling solutions into these areas, always focus on the three R's of mission-critical infrastructure design concepts: reliability, redundancy and resiliency. This means you will not be looking for the cheapest solution or the areas in which to cut corners. Selling mission-critical solutions means selling resiliency, redundancy and reliability as a total package, not discounting one or all of them to come up with a cheaper price.
Are power failures on the rise, or are design specs being lowered?
Are we experiencing more power failures for some unknown reasons, or are some in charge of designing network and power infrastructures trying to save some money by cutting corners when it comes to designing, implementing and managing facilities supporting mission-critical applications and operations? This is a question that needs an answer soon, because all infrastructure is being subjected to more use, stress and abuse on a daily basis.
Are you raising these questions and concerns about the importance of redundancy, reliability and resiliency with your corporate and government customers? You should be.