Fueling a Data Center Revolution

shutterstock / AURIELAKI. Data center, fuel cells, and microgrid.

Fuel cells have the potential to power a large data center, free it from utility connections, save energy and ensure reliability.

With McKinstry Electric, the electrical contractor that built and hosts the data center, Microsoft is monitoring the Advanced Energy Lab test site in Seattle.

The goal is to create a data center that could be completely independent of the power grid, but a lot of testing is ahead. The pilot was launched two years ago and will continue several more years, after which Microsoft may deploy fuel cells at other data centers around the world.

McKinstry installed the natural- gas-powered fuel cell system at its location in Seattle’s Sodo district to test the lower carbon-emission-based data center. The 2,000-square-foot site includes 20 racks of servers, with 10 fuel cells above the stacks, mounted on a steel frame supporting battery units, each about the size of a refrigerator.

By putting the power locally above each stack, the system requires no power production from an electrical utility.

Since the test site went live, Microsoft has been monitoring the results, said Sean James, Microsoft’s energy research director. So far, the company has found “the fuel cells systems have turned out to work very well with the servers.”

Microsoft is looking into other ways the system could be made more efficient.

“Our engineers are currently focused on finding ways to optimize the integration between the server and fuel cell and finding ways to reuse the waste streams,” James said.

McKinstry's Dan Ronco demonstrates energy features in the Advanced Energy Lab data center.
McKinstry's Dan Ronco demonstrates energy features in the Advanced Energy Lab data center. photo credit: McKinstry Electric.

For McKinstry, the $3 million project is just one of a series of contracts the company has had with its software-providing neighbor. This latest deployment is funded, in part, by a $674,000 grant from a Washington state program that aims to stimulate the development of clean-energy technology.

Microsoft’s efforts began several years before McKinstry built the Seattle test site.

Prior to building a data center laboratory, “we had to rigorously test the concept of in-rack fuel cells,” James said, adding that the company accomplished this through a partnership with the National Fuel Cell Research Center at the University of California, Irvine.

To demonstrate whether fuel cells could operate solely on clean gas, Microsoft tested a fuel-cell based power source for a mock-up of a data center in Cheyenne, Wyo., using a renewable fuel—biogas—in 2014. The project began with evaluation of the natural gas engine generators as primary power. Cheyenne was selected as the test site due to its high elevation. The site required supertankers of fuel, and the high altitude provided extra challenge for engine performance.

McKinstry set up and managed the generator testing in Cheyenne. That effort, after multiple rounds of testing and innovating, proved the technology worked as needed, said Dan Ronco, McKinstry’s director of electrical construction.

Then, the team went to work on both the biggest opportunity, and greatest challenge yet: “Taking the fuel cell concept from the training bench to an actual data center so we could test and learn from it in a real environment,” James said.

Construction on the mock-up data center began in March 2017, after a year-long development and design process, Ronco said. A ribbon-cutting ceremony took place in October 2017.

Since then, McKinstry and Microsoft have been testing the efficacy and functionality of using fuel cells to power that data center, he said.

McKinstry Electric, a national leader in high-performing building design, construction, operations and maintenance, is both a bid and design/build firm. Its expertise is in streamlining the kind of creative design process that was needed for this kind of challenge. The company also provides technical deployments for both mechanical and electrical systems, making it a one-stop shop for such a project. For Microsoft, that meant collaborating with one company rather than several to accomplish the electrical and mechanical installation.

Much of the specialized structure for the data center was fabricated on-site. Design took more than a year, with incentive to get it done expediently. Microsoft wanted to gain insight as quickly as possible.

“Time was of the essence; the sooner the technology company can prove the system, the sooner they could expand it,” Ronco said.

Although they were building a small piece of a large data center, McKinstry designed it to look and feel like a regular data center, Ronco said. Test data streaming through the center consists of nonsensitive information.

“Demonstrating that fuel cells could simplify the energy supply chain has been full of interesting challenges,” James said. “First, we had to craft a solution that enabled us to cut not one but several key pieces of electrical equipment, such as transmission lines.”

The natural gas that feeds into the fuel cells triggers an electrochemical reaction, leading to extraction of hydrogen atoms that send a current of electrons to power servers.

Fuel cells are not entirely new to data center power. Some fuel-cell systems already exist in data centers as a form of power redundancy. In such cases, the systems still have a power plant nearby and high-voltage transmission lines feeding distributed power to the racks.

Here, there is no backup. The center relies entirely on the fuel cells. If one fuel cell fails, the others can provide the necessary additional power from their own reserves.

Operating with fuel cells meant the design had to accommodate a few unique challenges. For instance, the fuel cells vent high-temperature exhaust. So the system needed to provide a common way to vent that heat, in the same way a standard data center would vent heat, without adjusting the layout of racks and servers.

Servers are stored in racks below fuel cells at the Advanced Energy Lab. photo credit: McKinstry Electric.
Servers are stored in racks below fuel cells at the Advanced Energy Lab. photo credit: McKinstry Electric. 

To vent the heat, McKinstry designed the heat exhaust system to be as conventional as possible. For that reason, the fuel cells vent into a hot aisle and then outside, like in a traditional data center. In the future, the team may engineer ways for some of that hot air to be used for other purposes, such as creating more electricity.

Additionally, the system offers tools for testing. For instance, server functionality needs to be tracked to see how they operate over time. To accomplish this, “We put in a very sophisticated monitoring system, with 500 data points being monitored,” Ronco said, which includes life safety, air quality, efficiency of fuel cell, and changes in fuel supply.

While McKinstry designed a system to capture the data, the center relies on a Siemens software package on a common dashboard to extract the necessary information. Washington state helped fund this monitoring package to track and study all facets of the center’s operation.

Engine and power generation company Cummins developed a specialized inverter that takes DC voltage power from the fuel cell and converts it to AC voltage to go into the servers. Getting fuel cells to work with each other was a challenge, because they are not designed to do so. Cummins put together a custom inverter that made it feasible for fuel cells to operate together, which McKinstry installed with a fuel-cell connection.

The company developed this type of inverter for data center applications, and it was customized to meet the project’s power and reliability requirements.

“The partners worked together to build this data center using Cummins’ experience in power systems, inverters and hybrids across many applications to meet the unique objectives and requirements,” Ronco said.

McKinstry also installed a de-ionization skid to provide the cleanest water possible, which reduced the need for fuel-cell maintenance. They also installed a gas analyzer to extract information about the characteristics of gas and exhaust from the fuel cell. In this way, the team can better understand what happens inside, what the fuel looks like going in, and what the exhaust coming out does.

Monitoring efficiency is another part of the system. Traditional generator efficiency is merely 25 percent, while fuel cells are in the range of 60-plus percent and could be as high as 80.

“One of our main goals was to prove that this was safe and achievable,” Ronco said. “[McKinstry] had to devise a system that could meet FC-1, a 700-page ANSI standard. [There was] definitely lots of engineering headaches to make sure what we ended up with would pass those evaluations and prove it’s safe.”

The fuel-cell units themselves are like big, heavy refrigerators. Sodo, in Seattle, is in a seizmic zone; therefore, installing them to be earthquake-safe required specialized secure mounting. However, it also needed to be disassembled without removing bolts or welds so Microsoft could quickly disconnect for maintenance and repair. McKinstry devised and fabricated a pin-and-sleeve disconnect skid in its fabrication shop.

“The objective of our Advanced Energy Lab pilot is to trial the effectiveness of powering a data center with fuel cells, simplifying the long transmission process that afflicts traditional data centers,” James said. “Using fuel cells reduces the number of transmission steps, lowering the cost of data center operation and allowing less opportunity for unreliability, while almost doubling the data center’s energy efficiency.”

“Microsoft’s data centers are already as much as 93% more energy-efficient and as high as 98% more carbon-efficient than on-premises data centers,” James said, adding that it’s due to extensive efforts in IT efficiency from chip-to-data-center infrastructure and renewable energy. “With strong investment in R&D projects like the Advanced Energy Lab, we hope to take this a level further.”

James said the Advanced Energy Lab could also pay dividends for other large consumers of energy,” such as electric utilities, who are increasingly tasked with bringing more energy into cities, despite the challenge of limited space for more electrical lines.

“It’s a very safe, secure but sophisticated operating system,” Ronco said, adding that there’s no potential chance for gas levels to get out of balance. “Because we have taken upon ourselves to invest in design and construction to include all the specialty systems, architectural metals, it put us in a position to help innovate.”

The data center has garnered accolades, winning several industry awards.

Stay Informed Join our Newsletter

Having trouble finding time to sit down with the latest issue of
ELECTRICAL CONTRACTOR? Don't worry, we'll come to you.