In this post I will go through a few questions around hybrid Linux/Windows clusters and try to clarify their purpose and their applicability. Before going into the details of this topic, I would first address the question:
Why would one even consider a hybrid cluster?
The drive behind the current software delivery methodologies has been the development of containerization technology.
Whether Linux fans like myself like it or not, to this day, Windows powers the vast majority of business applications, with about 72% of servers worldwide. Windows domination of the market is a historical trend which predates the containerization techniques which revolutionized software delivery. Now Windows legacy applications are faced with the need to adapt to these techniques in order to keep a competitive edge.
To facilitate this task, Windows has moved towards the adaptation of containerization technology such as a Docker for Linux and Windows containers and, more recently, Kubernetes. Currently, management nodes in Kubernetes can be only in Linux, but a cluster can include one or more Windows workers. Therefore, Windows K8S clusters are, by definition, hybrid clusters.
In theory, Windows based applications could be containerized in Windows containers, and be delivered and deployed in a K8S cluster and benefit from this technology like their Linux counterparts, without significant changes to their code. However, the implementation of this idea is not as seamless as it sounds.
Why a hybrid cloud should not be the end goal?
Despite the significant efforts, the containerization technology for Windows is not yet mature to run in production. Moreover, the maintenance of a hybrid cluster comes with significant challenges. For instance, there might be differences between the Linux and the Windows containerization which cannot be transparent to the user. This doubles the maintenance work and the knowledge needed to manage a hybrid cluster. Furthermore, existing components were not conceived to accommodate a hybrid cluster and, therefore, they might not be available for Windows nodes, and if they are, in many case, it is just in alpha versions.
Given these drawbacks, can we justify the use of hybrid clusters?
For a legacy Windows application, the current situation can evolve in two directions: either the technology of Windows containerization matures and becomes production-ready, or it continues to be unstable, resulting in a decision to move to Linux containers.
In the first scenario, whoever opted for a hybrid cluster would have already gathered knowledge and paved the path towards a Windows native containerization. Therefore, the initial investment would payback in terms of few or no efforts to adapt in the future.
In the second scenario, a hybrid cluster allows a gradual transition from a full Windows container environment, to hybrid Linux/Windows container, and finally to full Linux containers, while continuously validating the deployment and the transition process. Therefore, I would argue that, in any situation, having a hybrid cluster is a great way to continuously deploy and test a Windows based application until a more stable environment is available.
What is the most straightforward way to bring up such a cluster?
During my first attempts to build a hybrid cluster, I used vanilla K8S and tested with Calico and Flannel CNIs. I faced various challenges, including network, runtime and volume issues, but despite this, I managed to have a simple hybrid cluster. The real drawback of these clusters was the “day two”, the long-term maintenance. Small changes in infrastructure, such as a node being unavailable, would put the system, in an inconsistent state and repairing it was unsustainable. Therefore, I gave up vanilla K8S, for Rancher’s RKE2, which is not only a great K8S distribution on its own, but also have a great support for hybrid clusters.