Data center power consumption is a critical issue faced by both small and large organizations. Underscored by estimates Lawrence Berkeley National Laboratory that data centers in the United States were to have consumed 70 billion kilo-watt hours (kWh), or about 1.8% of total U.S electricity consumption. The impact of growing power consumption has forced organizations to allocate valuable CAPEX in more efficient data center technologies.
Data center managers face the daily challenge of managing power consumption and its impact to OPEX and CAPEX. Global data centers measure their power capacity and consumption in megawatts, not watts. Thus incremental, “one-watt-at a-time” strategies may seem like a small drop in the bucket, but given time and scale, they can reduce power consumption.
Critical load generated by IT equipment, such as servers, routers and switches are only part of the power equation. Ancillary power for cooling, lighting, and other equipment must also be included in the data center power consumption equation. Not all that long ago, data centers planned on a 1-to-1 ratio between IT critical load and support power. Meaning that for every dollar spent on critical load, one dollar was required for support power. This model did not scale well to the era of massive cloud data centers.
Overall, today’s data centers tend to operate with efficiency metrics between 40% and 70% of the critical load. Meaning that for every 1 watt of IT critical load used, an additional 0.4W to 0.7W of support load is used to cool and maintain the data center. Given these data center economics, reducing the IT load by 1 watt can really mean reducing to total power load by 1.4 to 1.7 watts.
Max Power Consumption | SFP+ 10G | QSFP+ 40G | QSFP28 100G |
DAC (one end) | 0.1W | < 0.5W | < 1W |
Short Reach Transceiver (each) | 0.9W | 1.5W | 3.0W |
That is where looking at network cabling power offers opportunities for incremental efficiencies. Current data center trends toward top-of-rack switching are driving efficiencies one watt at a time. Conventional top-of-rack architectures use direct attached cables, or DACs, to connect servers in a rack to a switch in each rack. DACs are pre-terminated copper cable assemblies with a transceiver pre-terminated on each end and installed into the SFP+ or QSFP ports in a switch or server. In addition to being lower cost than a transceiver and structured cabling solution, DACs offer efficiencies by not performing optical to electrical conversion inside the module. The cost savings from not operating a laser offers, at bare minimum, 70% power savings over a short reach transceiver. Case in point of a 10G SFP+ DAC cable. The combination of both ends requires 1.5W less power than two 10G SFP+ SR transceivers. 40G and 100G connections see even larger power consumption advantages with DAC cables.
Applying these numbers to the real world, we need to answer the question, “In a world with 10-megawatt data centers why should data center managers care about 1.5 watts?”
Like compounded interest on money, power savings in the data center also compounds. As we include the power efficiency metric of the data center, we also save on support power in the data center. In this example, for each 1.5W of IT power savings, another 0.6 to 1.05 watts of support power is saved. Offering a net savings of 2.1W to 2.55 watts saved per connection. A data center with a top-of-rack architecture and 1000 cabinets could realize up to 21kW power savings using DAC cables.
The net effect on operating expenses comes in calculating the kilowatt hour savings multiplied by the cost per kilowatt hour.
The cost of electricity may vary by up to 200% depending upon country and season. The June 2018 U.S, Energy Information Administration proves out this assumption, data centers with higher power costs stand to gain the most from top-of-rack DAC connections.
Region | Average Industrial Utility Rate per kWh June 2018 | Savings ($) |
New England | $ 0.12 | $ 23,631.02 |
Mid Atlantic | $ 0.07 | $ 13,050.25 |
West North Central | $ 0.08 | $ 15,253.79 |
Mountain | $ 0.07 | $ 13,031.26 |
Pacific Contiguous | $ 0.11 | $ 20,344.72 |
South Atlantic | $ 0.07 | $ 12,613.34 |
East South Central | $ 0.06 | $ 11,625.55 |
East North Central | $ 0.07 | $ 13,240.21 |
West South Central | $ 0.06 | $ 10,675.75 |
Pacific Non-Contiguous | $ 0.24 | $ 45,267.47 |
Very few of today’s massive data centers are deploying 40G and 100G for top-of-rack connections. However, those that do or may plan to stand to realize even greater savings!
40G QSFP+ | 100G QSFP28 | ||
DAC Savings/ connection | 2 | 4 | W |
Power Efficiency | 0.4 | 0.4 | |
Savings per connection | 2.8 | 5.6 | |
10 connections/rack | 28 | 56 | W |
1000 cabinets | 28 | 56 | kW |
kWh/year | 175200 | 490560 | kWh |
Savings by Region | |||
New England | 21,794.88 | 61,025.66 | $ |
Mid Atlantic | 12,036.24 | 33,701.47 | $ |
West North Central | 14,068.56 | 39,391.97 | $ |
Mountain | 12,018.72 | 33,652.42 | $ |
Pacific Contiguous | 18,763.92 | 52,538.98 | $ |
South Atlantic | 11,633.28 | 32,573.18 | $ |
East South Central | 10,722.24 | 30,022.27 | $ |
East North Central | 12,211.44 | 34,192.03 | $ |
West South Central | 9,846.24 | 27,569.47 | $ |
Pacific Non-Contiguous | 41,750.16 | 116,900.45 | $ |
Reducing data center power consumption one watt at a time with direct-attached cables is not a futile approach in the race against rising power consumption. The cost models show real-world OPEX savings over transceiver and structured cabling alternatives.