Borg: Kubernetes’ Predecessor
It’s 2024, with a little bit of stretch…it’s been nearly a decade since Kubernetes was released. More exactly Kubernetes 1.0 was released on July 21, 2015 [1]. Google worked with the Linux Foundation to form the Cloud Native Computing Foundation (CNCF)[2] and offered Kubernetes as a seed technology.
Kubernetes traces its lineage directly from Borg. Many of the developers at Google working on Kubernetes were formerly developers on the Borg project [3].
Borg was a unified container management system, built to manage long-running processes and batch jobs. The resource isolation provided by containers has enabled Google to drive utilization significantly higher than industry norms at that time.
Architecture
A Borg cell consists of a set of machines, a logically centralized controller called the Borgmaster, and an agent process called the Borglet that runs on each machine in a cell. All components of Borg are written in C++.
Borg stored the state of the cluster in a centralized Paxos-based transaction-oriented store that was accessed by the different parts of the cluster control plane, e.g. schedulers, using optimistic concurrency control to handle the occasional conflicts.
“Borg’s users are Google developers and system administrators, that run Google’s applications and services. Users submit their work to Borg in the form of jobs, each of which consists of one or more tasks that all run the same program (binary). Each job runs in one Borg cell, a set of machines that are managed as a unit.”[4]
So in a nutshell, Borg hides the details of resource management and failure handling so its users can focus on application development instead.
Although Borg’s primary role is to manage the lifecycle of tasks and machines, the applications that run on Borg benefit from naming and load balancing. Similarly, Kubernetes supports naming and load balancing using the Service abstraction.
A Borg alloc (short for resource allocation) is a reserved set of resources on a machine in which one or more tasks can be run, in Kubernetes, it is called a Pod.
An intermediate cluster management system was Omega[5], an offspring of Borg, and many of Omega’s innovations (including multiple schedulers) have been folded into Borg.
One of Kubernetes' significant improvements compared to Borg is the fact that its architecture is purely API-driven, “enforcing” all store accesses through a centralized API server that hides the details of the store implementation and provides services for object validation, and versioning.
The cornerstone of Kubernetes is the reconciliation loop, which has at its core the controller pattern that helps in achieving and maintaining the desired state.
The Kubernetes API makes a distinction between the specification of the desired state of an object (a nested object field called spec) and the current status of the object (a nested object field called status).
In conclusion, years of expertise in building container management systems have taught Google much, and many of those lessons are embedded into Kubernetes.
References
[1] https://en.wikipedia.org/wiki/Kubernetes#cite_note-17
[2] https://en.wikipedia.org/wiki/Kubernetes#cite_note-18
[3] Kubernetes blog
[4] Borg Whitepaper
[5] Omega
