Q 8:     Has breaker coordination been done in conjunction with the arc flash analysis?

This seems like a question with an obvious answer, but it is not.  Arc flash has become a major concern in data centers due to the high power levels that are now required.

A legitimate arc flash hazard analysis first requires a thorough short circuit study which, although a standard requirement of the electrical design, must be done with a different understanding when the intent is to also reduce the arc flash hazard. The breaker coordination study must be done with the same goal in mind. If all three of these studies are done together, and in the right order, they could result in the selection of different circuit breakers or fuses than might otherwise have been chosen. Since the destructive discharges in this project are being reported as resulting from fault conditions, it becomes logical to ask if there are flaws in either the way these studies were performed, in the protective devices that were selected, or in their settings.

Q 9:     Have Surge Protection Devices been included at all necessary points?

Surge protection is required by code at the service entrance.  But best practice today dictates installing surge protection devices (SPD’s) on every breaker panel in the power chain serving a data center, including those after the UPS.  This is particularly important on UPS bypass circuits.  SPD’s (formerly known as TVSS’s or Transient Voltage Surge Suppressors) come in different types and must be selected for the conditions present at each point of power control. The best choices for data centers are hybrid designs that handle high currents but are also very fast acting when necessary. Depending on the type of SPD selected, and the characteristics of the power surge, the SPD may “self sacrifice” in order to protect the equipment following it. But regardless of the selection, unless an SPD is properly installed it will not serve its intended purpose. Wires must be as short as possible and, like grounding installations, they must be radiussed if they have to be bent at all. It’s even best to use special conductors to ensure the lowest loss of high-frequency transients and maximum performance from the SPDs.  Was this all properly done at the NSA data center? If so, what kinds of surges could be getting past them and damaging equipment?

It is certainly recognized that whatever has gone wrong at the NSA facility has not resulted from anything so simple as the answer to any one of these questions. It is more likely that, directly or tangentially, some combination of the occurrences questioned here has caused or contributed to the problem. Regardless, when we examine projects on which we might want to bid, and when we design projects we have been successfully awarded, we should keep all these items in mind along with many others on the cooling and structural sides that may not have even been encountered yet at NSA. Doing so could go a long way to ensuring that we aren’t the next subject of negative headlines in the Wall Street Journal and all over the web.