A New Approach To Building Data Centers
In northwest Phoenix, Aligned Data Centers is building a massive facility that will use substantially less power and water due to an innovative design and construction plan.
Most conspicuously, this data center does not have the traditional raised floor seen in many centers throughout the country. Raised-floor systems deliver cooled air through perforated floor tiles. Other data centers cool equipment through cooled air overhead.
The 550,000-sq-ft Aligned Data Center—as well as a smaller, completed facility in Plano, Texas—uses Aligned Energy’s modularized infrastructure system that has been built into the facility, including patented eSYNC heat removal systems and power. As each cabinet is populated, the above-rack heat absorption system is connected to immediately normalize IT loads.
“When data centers were first starting to be built, teams took what they knew about office buildings and built a data center,” says Jakob Carnemark, CEO of Aligned Energy, parent company of Aligned Data Centers, and an ENR Newsmaker in 2013.
Those early cooling models included chilled water plants, which Carnemark says are capital intensive and use massive amounts of energy. Aligned Energy’s approach employs a heat sink, which draws the hot air from the servers, then passes it across coils to neutralize it, resulting in a 75° to 77° F ambient environment.
“It’s purely a heat removal cycle. It’s not designed for human cooling,” Carnemark says.
The result is a data center that will operate on up to 80% less energy for cooling and use up to 85% less water compared with a traditional chiller system. For example, power is about 20% to 40% of the IT load in a chiller plant/forced air system but about 5% of IT load in Aligned Energy’s heat rejection system.
The project is also an exercise in modularity for increasing efficiencies thanks to the decentralized cooling approach. The white-space room, which houses servers, is surrounded by a central access point comprised of a multilevel system distribution spline that includes both active power, back-up power and heat rejection components. These central access points have been termed “thermal bus” by Aligned Energy. A steel exoskeleton was built around the central supply area with a roof-level concrete deck that can support roof-mounted generators and a large gantry crane for equipment changes.
Prior to 2013, Carnemark was a Skanska senior vice president for mission critical projects, a position he held for about a dozen years. In that time, he and his teams built several large data centers before realizing there was a better way, he says. Initial pilot testing of the concept and creating the Inertech system—now owned by Aligned Energy—was led by Skanska at the TELUS Super Data Center in Rimouski, Quebec, from 2010 to 2012. The proprietary modular cooling system was developed by a firm called Inertech, which Aligned Energy also owns.
Other members of the Aligned Energy staff have also come from Skanska, including Steven Del Toro, director of construction and real estate. The result was an owner with precise construction ideas and a strong vision of a more efficient data center.
“We are coming at it from a technological angle to solve data center problems and meet the future demand that will be required to keep our business growing,” says John Petralia, chief marketing officer at Aligned Energy.
Kevin Connor, project manager at Skanska, says because of the construction and data center experience of the Aligned Energy team, collaboration and sharing was extensive. Although the project was not delivered under the tenets of integrated project delivery, he says that like most data center projects, the process was an exercise in working together.
“When building data centers, most owners are specialized in designing and building it. Working around people who know what they are talking about is par for the course,” he says.
The Aligned Energy team served as the principal factor behind design and contributed to solving construction-related issues. More than 59 subcontractors worked on the project. Maximum daily worker count was 250 for about 200,000 worker hours.
The structure was originally built for Honeywell more than 30 years ago and had been vacant since 2008. Past uses included fabricating and assembling automated factory components, and Connor says ductwork, catwalks, bathrooms and managerial areas were gutted as part of early prep work.
Del Toro says the building itself had “great bones,” allowing for minimal shell improvements. The facility design resembles an H—there are two large and longer bays that surround a shorter bay in the middle. The centrally located thermal bus within this area is one part of the larger approach of locating power and emergency back-up on the roof.
While the shell was strong, its roof was not built to support equipment and machinery. The Aligned Energy design team’s goal of centralizing the facility’s systems distribution meant constructing vertically while minimizing any potential impact to valuable real estate below.
The top level of the mechanical and electrical spline required reinforcement. With help from subcontractor SME Steel, an exoskeleton was erected around the center bay and a concrete structural deck for the generators and other roof-top equipment was assembled utilizing a 386-ton crane. Building the exoskeleton took about four months and nearly 1,100 tons of steel. The concrete pad is approximately 160,000 sq ft and was under construction for three weeks.
“Who else has a levitating equipment platform over the building?” Connor says.
A gantry crane will be installed at a later stage so roof-top generators can be swapped if necessary. At construction’s end, most of the steel exoskeleton and equipment will be hidden from view by a fabric shell.
“It started out with a creative concept design and preliminary loading analysis, and ultimately there was a straightforward design of new foundations that were created inside the footprint” Connor says.
The steel exoskeleton was constructed with a cantilever design using a truss system where the superstructure above the equipment acts like tension rods, Connor says.
Much of the interior construction work surrounded the piping system. The air-cooled adiabatic rejection system, called Cactus by Aligned Energy, comprises a fluid cooler with an indirect evaporative cooling mode and a compressor trim unit that rely on less than 10 moving parts. Condenser heat rejection and water-side functions are combined, allowing Aligned Energy to guarantee a 1.15 annualized power usage effectiveness.
Carnemark says the systems use refrigerants 1234ZE or R134A but can also “use the other new fluids being developed that have low global warming potential.”
The refrigerant and compressor trim components will be able to be installed as capacity at the data center grows and can also be deployed in little increments conforming to 30-kW units in order to “right size” deployment of cooling infrastructure in line with IT demand for the facility. Connor says piping will be completed for full capacity at build-out.
As with most modern data centers, an upstream substation feed is required. Aligned Energy and local utility Arizona Power Service reached an agreement to jointly construct an onsite dedicated substation with multiple incoming medium voltage diverse feeds. In order to further coordinate incoming power, a microgrid agreement was also developed to potentially support the existing electrical grid upstream, if necessary. The generators can be used to provide back-up power to the data center as well as proactively brace the data center during times of grid instability to further enhance reliability.
“The upside of having the partnership with APS is they built [the substation] really fast,” Connor says.
At the heart of the microgrid is the 69-kilovolt substation. Four redundant power feeds from three upstream sources bring electricity to the substation for built-in utility resiliency. The uninterrupted power system protects the entire facility because the system is above the circuit breakers. Carnemark says this allows for a more efficient system that provides a better guarantee of reliability but also helps manage load swings from the grid and substation, also known as “ripples.”
Aligned Energy is not just developing economical data centers in order to be good ecological stewards and to lower costs in the U.S., where energy is plentiful, Carnemark says. The company also wants to test how such systems can be duplicated in areas with less-capable infrastructure systems.
“Data centers are going to grow in other parts of the world that don’t have the U.S. grid,” he says. “You can’t leverage traditional data center infrastructure there.”
According to Connor, who led some of the initial builds in Canada and leads the construction work for the Phoenix deployment, the fact that the concept has become a reality is impressive.
“I have been able to see it come from a concept,” Connor says. “It worked exactly like they told us it was going to work.”