Google is known to fiercely guard the secrets of its data centers, but Facebook is not. The social media giant has released two important tools that it uses internally to leverage its huge social network as open source code.
The company has released Katran, the load balancer that prevents overloading of corporate data centers, as open source under the GNU General Public License v2.0 and available on GitHub. In addition to Katran, the company offers details about its Zero Touch Provisioning tool, which it uses to help engineers automate much of the work required to build its backbone networks.
This is not Facebook’s first foray into open source software that manages its network. Last month, the company opened PyTorch, the software used for its artificial intelligence (AI) and machine learning projects. PyTorch is a Python-based package for writing tensor calculations and deep neural networks using GPU acceleration.
Facebook needs to develop this type of software package because while there are many software products off the shelf, none of them are designed for a global social media company with 2 billion users.
Facebook Load Balancing Tool Details
The news came from a blog post written by Facebook production engineer Nikita Shirokov and software engineer Ranjeeth Dasineni. Both said the company previously made its own load balancing software, mostly from open source software. He served them well for four years, but he was starting to show his age and his limitations.
They wrote that a load balancer must meet four criteria: it must run on basic Linux servers; coexist with other services on a given server, thus eliminating the need for dedicated load balancing servers; allow maintenance without disruption; and offers easy instrumentation and debugging.
Shirokov and Dasineni said their first software-defined load balancer, called Layer 4 Load Balancer or L4LB for short, was insufficient in terms of coexistence with other service criteria, especially backends.
“In the second iteration, we used the eXpress Data Path (XDP) framework and the new BPF virtual machine (eBPF) to run the software load balancer with the backends on a large number of machines,” they wrote.
Facebook Zero Touch Provisioning Tool Details
Details behind Facebook’s Zero Touch Provisioning tool also came in the form of a blog post written about three weeks ago by a number of Facebook engineers. Contactless provisioning (ZTP) allows you to automatically provision new switches and routers in your network, with no manual intervention required.
For a business the size of Facebook, it needs to build its own networks, which is why it needs a ZTP tool. And like load balancing, he found existing ZTP tools to be inadequate to handle the kind of scale Facebook operates on.
“Ultimately, these challenges prompted Facebook’s network engineers to develop a whole new approach to network deployment workflows,” the blog’s authors said.
So, he created a new framework called Vending Machine, but here instead of popping in a dollar and getting a can of soda, the entry is a role, location, and device platform – and a Freshly provisioned network device appears, ready to deliver production traffic. ZTP is always evolving and Facebook is adding new features to it.
Facebook did not disclose the license for ZTP. And like Katran, if you decide to use it, you are on your own. Some questions may be answered by a friendly Facebook engineer, but this software is not supported. So, I’m curious to see who could actually use this software, as it’s not exactly SMB hardware. It is aimed at large companies, and I think most of them have their own platforms. But I could be wrong.
Copyright © 2018 IDG Communications, Inc.