Server Generations

Freedom laid the groundwork for new generations to come. In 2012, Facebook released the specs for its second server design, dubbed "Windmill", which brought updated mainboards supporting Intel's Sandy Bridge-EP platform and AMDs Opteron 6200/6300 CPUs.

Following table details all Facebook-designed OCP servers, starting with Freedom, with highlighted differences.

Facebook-designed OCP Server Generations
  Freedom (Intel) Freedom (AMD) Windmill (Intel) Watermark (AMD) Winterfell Leopard
Platform Westmere-EP Interlagos Sandy Bridge-EP Interlagos Sandy Bridge-EP / Ivy-Bridge EP Haswell-EP
Chipset 5500 SR5650/
SP5100
C602 SR5650/
SR5670/
SR5690
C602 C226
Models X5500/
X5600
Opteron 6200/6300 E5-2600 Opteron 6200/6300 E5-2600 v1 / v2 E5-2600v3
Sockets 2 2 2 2 2 2
Max TDP allowed (in Watt) 95 85 115 85 115 145
RAM per socket 3x DDR3 12x DDR3 8x DDR3 8x DDR3 8x DDR3 8x DDR4
/NVDIMM
~ Node Width (inch) 21 21 8 21 6.5 6.5
Form factor (height in rack U) 1.5 1.5 1.5 1.5 2 2
Fans per node 4 4 2 4 2 2
Fan width (in mm) 60 60 60 60 80 80
Disk bays (3.5'') 6 6 6 6 1 1
Disk interface SATA II SATA II SATA III SATA III SATA III / RAID HBA SATA III
/ M.2
Amount of DIMM slots per socket 9 12 9 12 8 8
DDRX gen 3 3 3 3 3 4
Ethernet connectivity 1 GbE fixed 2 GbE fixed 2 GbE fixed + PCIe mezz 2 GbE fixed 1GbE fixed + 8x PCIe Mezz 8x PCIe
Mezz
Deployed in Prineville, Oregon Prineville Lulea, Sweden Lulea Altoona ?
PSU model PowerOne SPAFCBK- 01G PowerOne SPAFCBK- 01G PowerOne PowerOne N/A N/A
PSU count 1 1 1 1 N/A N/A
PSU capacity (in Watt) 450 450 450 450 N/A N/A
Amount of nodes per sled 1 1 2 2 3 3
BMC No (Intel RMM) No No (Intel RMM) No No (Intel RMM) Yes (Aspeed AST1250
w 1GB
Samsung DDR3 DIMM K4B1G1646G- BCH9 )

Facebook indicated that its expected lifespan for its Freedom nodes is around three years, and is in the process of swapping out deprecated equipment with OpenRack v1 based equipment.

Introduction Integrate: OpenRack
POST A COMMENT

27 Comments

View All Comments

  • Black Obsidian - Tuesday, April 28, 2015 - link

    I've always hoped for more in-depth coverage of the OpenCompute initiative, and this article is absolutely fantastic. It's great to see a company like Facebook innovating and contributing to the standard just as much as (if not more than) the traditional hardware OEMs. Reply
  • ats - Tuesday, April 28, 2015 - link

    You missed the best part of the MS OCS v2 in your description: support for up to 8 M.2 x4 PCIe 3.0 drives! Reply
  • nmm - Tuesday, April 28, 2015 - link

    I have always wondered why they bother with a bunch of little PSU's within each system or rack to convert AC power to DC. Wouldn't it make more sense to just provide DC power to the entire room/facility, then use less expensive hardware with no inverter to convert it to the needed voltages near each device? This type of configuration would get along better with battery backups as well, allowing systems to run much longer on battery by avoiding the double conversion between the battery and server. Reply
  • extide - Tuesday, April 28, 2015 - link

    The problem with doing a datacenter wide power distribution is that at only 12v, to power hundreds of servers you would need to provide thousands of amps, and it is essentially impossible to do that efficiently. Basicaly the way FB is doing it, is the way to go -- you keep the 12v current to reasonable levels and only have to pass that high current a reasonable distance. Remember 6KW at 12v is already 500A !! And thats just for HALF of a rack. Reply
  • tspacie - Tuesday, April 28, 2015 - link

    Telcos have done this at -48VDC for a while. I wonder did data center power consumption get too high to support this, or maybe just the big data centers don't have the same continuous up time requirements ?
    Anyway, love the article.
    Reply
  • Notmyusualid - Wednesday, April 29, 2015 - link

    Indeed.

    In the submarine cable industry (your internet backbone), ALL our equipment is -48v DC. Even down to routers / switches (which are fitted with DC power modules, rather than your normal 100 - 250v AC units one expects to see).

    Only the management servers run from AC power (not my decision), and the converters that charge the DC plant.

    But 'extide' has a valid point - the lower voltage and higher currents require huge cabling. Once a electrical contractor dropped a piece of metal conduit from high over the copper 'bus bars' in the DC plant. Need I describe the fireworks that resulted?
    Reply
  • toyotabedzrock - Wednesday, April 29, 2015 - link

    48 v allows 4 times the power at a given amperage.
    12vdc doesn't like to travel far and at the needed amperage would require too much expensive copper.

    I think a pair of square wave pulsed DC at higher voltage could allow them to just use a transformer and some capacitors for the power supply shelf. The pulses would have to be directly opposing each other.
    Reply
  • Jaybus - Tuesday, April 28, 2015 - link

    That depends. The low voltage DC requires a high current, and so correspondingly high line loss. Line loss is proportional to the square of the current, so the 5V "rail" will have more than 4x the line loss of the 12V "rail", and the 3.3V rail will be high current and so high line loss. It is probably NOT more efficient than a modern PS. But what it does do is move the heat generating conversion process outside of the chassis, and more importantly, frees up considerable space inside the chassis. Reply
  • Menno vl - Wednesday, April 29, 2015 - link

    There is already a lot of things going on in this direction. See http://www.emergealliance.org/
    and especially their 380V DC white paper.
    Going DC all the way, but at a higher voltage to keep the demand for cables reasonable. Switching 48VDC to 12VDC or whatever you need requires very similar technology as switching 380VDC to 12VDC. Of-course the safety hazards are different and it is similar when compared to mixing AC and DC which is a LOT of trouble.
    Reply
  • Casper42 - Monday, May 4, 2015 - link

    Indeed, HP already makes 277VAC and 380VDC Power Supplies for both the Blades and Rackmounts.

    277VAC is apparently what you get when you split 480vAC 3phase into individual phases..
    Reply

Log in

Don't have an account? Sign up now