EXPERT OPINION: Rethinking the electricity distribution in datacentres to reduce the risk of outages
In 1893, Westinghouse Electric set about deploying the first electric infrastructure in the USA, using the alternating current (AC) electricity supply system recommended by scientist Nikola Tesla. Although it may sound surprising, the way electricity is distributed hasn't changed much since then, and the same goes for datacentres, which have changed very little since their much more recent beginnings. Of course, web hosting providers, to improve the availability of their infrastructures, make certain network components in the electricity supply chain redundant, but they've kept the general principle. How can we now optimise the electricity distribution in datacentres, which is currently between 1.5 and 2% of the world's energy consumption? What are the difficulties? How can these changes reduce the risk of outages? Germain Masse, Director of OVH Canada, offers his opinion.
Economic incentive versus reducing "failure domains"
When designing a datacentre, you're faced with a dilemma. One on hand, you may wish to divide up the electricity distribution as much as possible by multiplying the number of "small" transformers and UPSs to reduce the impact of a hardware failure. But this strategy increases installation and maintenance costs, and ultimately adds to the end-user's bill. Today, you can generally reach equilibrium by deploying transformers, 1 to 2 megawatts each, to which UPSs with capacities of 400 kilowatts up to 1 Megawatt are connected to the output side (take note that current generation UPSs are often made up of 200 to 400 kW modules). Because of their huge electrical power and criticality, these UPSs force us to accept fairly large failure domains, in the range of 3,000 to 6,000 machines. (UPSs are triggered into action when a power outage occurs, giving time to migrate to an emergency electricity supply and/or to start up electricity generators. This can take between 15 seconds and 1 minute).
This is one of the biggest challenges of datacentres, and is what we at OVH call a "failure domain". If a hardware component fails, all the impacted servers are located in the same failure domain. Depending on the architecture in place, some users will prefer to group servers in the same failure domain (for example in the event where several servers need to communicate with one another), or, on the contrary, divide the machines into separate failure domains (necessary for a disaster recovery plan, for example). To meet this kind of need, OVH lets users choose in which datacentre location their servers are located (in Europe: Roubaix, Gravelines or Strasbourg, or in Canada: Beauharnois). We want to go even further by allowing our customers to select their server rack. This will be a logistical challenge, but a huge advantage for our users.
And what if the alternative was direct current (DC)?
Across the globe, electrical current is supplied in AC. By contrast, components of a computer server (processor, disks) are powered by DC ranging from 3.3, 5 or 12 vaults, depending on the component.
When it arrives at a datacentre, electrical current is AC. It then undergoes various changes before it supplies the components of a server with DC. When we take a closer look, we can see what happens in a US datacentre:
You would think that you could remove the DC/AC and AC/DC phase between the UPS and the server. However, the electricity industry unfortunately doesn't move as quickly as the IT industry...
So why not supply datacentres directly with DC? When we look at the work of Thomas Edison, electricity was initially supplied in DC. To put it simply, in order to reduce loss in the flow of both AC and DC, the intensity of the current had to be lowered by increasing its voltage, as per the famous Joule effect. However, the voltage increase caused considerable energy losses, and so electricity companies decided to agree with Nikola Tesla and his AC electricity supply system.
Times have moved on. Now, with the help of electric power components (better known as convertors), we can change the electrical voltage of DC to transport it and limit energy losses. DC is in fact the preferred method for travelling very long distances, underground or underwater cables, etc., because it only requires two cables instead of AC's three.
The advantage of DC in datacentres, beyond simplifying the electricity supply chain by reducing the amount of hardware and limiting the risk of outages, is the possibility to combine various energy sources to (depending on the specific case) increase voltage, intensity and redundancy. The recent surge in new electricity production methods, such as solar power and fuel cells, is likely to push forward the evolution of DC; but there are two obstacles in the way. The first one is economic: DC hardware is currently more expensive than AC. The second is psychological: people don't know enough about DC operators...
UPS in the rack or on the server: which is the best strategy?
Whichever strategy you choose to move towards DC in the datacentre - transformation of AC to DC when the current reaches the datacentre, at UPS level, in the server room, the rack or the server - you need to change the way you power your servers.
As part of its contribution to the Open Compute Project, Microsoft recently announced plans to provide power to servers with lithium-ion batteries(1); an innovation that Microsoft calls Local Energy Storage (LES).
In reality, this isn't a new concept. In 2012
Supermicro announced batteries to power servers(2), 3 years after Google revealed that it used this method in some of its machines(3).
OVH's approach is original in the sense that instead of supplying power to the server, we use a simple DC/DC conversion step and, in every electrical rack, a hardware component (code name Altigo) receives AC to produce DC (24 or 48 volts, depending on the situation).
This solution enables us to use the same mass-produced batteries found in standard UPSs. Today this is much more economical than installing a battery on every server, although this may change if this new electrical power with batteries becomes commercially successful.
(1) Microsoft Reinvents Datacenter Power Backup with New Open Compute Project Specification
(2) Supermicro® Servers Offer Industry-First N+N+N Battery Backup Power (BBP™) Module Technology
(3) "Google's big surprise: each server has its own 12-volt battery to supply power if there's a problem with the main source of electricity."