Author: Hasham Haider

In most conversations about Kubernetes adoption you are likely to hear terms such as Day 1 and Day 2 operations. Day 1 and Day 2 operations encompass the mechanics of operating Kubernetes clusters once they have been installed, configured and made production-ready.

The first and most important question ITDMs need to ask is: Should we build Kubernetes from scratch, or subscribe to a managed service? If we subscribe to a managed service, which one will give us maximum flexibility and freedom, reduce our operational overhead and be cost-effective? We call this Day 0 of Kubernetes adoption.

The decision to build Kubernetes from scratch or subscribe to a managed service has implications for the operational costs and management overhead of getting Kubernetes production-ready. These cost and management overheads also spill over into Day 1 and Day 2 operations.

The “roll your own” Kubernetes path.

Building Kubernetes involves installing, configuring, managing, and operating Kubernetes on your own. The getting started documentation for Kubernetes does a great job of breaking down Kubernetes environments into components that need to be configured and managed separately by operations. These include cluster operations, cluster infrastructure, control plane, data plane and the applications running on top. Building out Kubernetes yourself will involve managing most, if not all, of these components in-house.

Another way of figuring out the operational and management overhead, and by extension the costs is to look at attributes of production-ready workloads. Production readiness is a fluid term. Karl Isenberg lists out the hard problems of Kubernetes:

• High Availability
• Single Sign On
• Multi-Tenancy
• Resource Isolation
• Permission Management
• Upgrades
• Backups
• Package Management
• CI/CD Integration
• Security
• Monitoring
• Troubleshooting

Getting all these ‘hard problems’ to a production-ready status requires considerable engineering time and effort. Platform9 estimates these costs to be approx $375,000 for the initial implementation. They also calculate yearly recurring operational costs to be approximately $ 687,500.

This is a considerable investment for scale-ups and SMEs to invest in tasks that could be outsourced to a managed service provider. The management overhead of setting up and continuously operating Kubernetes environments also eats into developer resources, which could be redirected to building a competitive advantage instead.

The Managed Kubernetes Path.

Managed Kubernetes services cover the spectrum, from basic Kubernetes cluster management tools for installing and operating clusters to full-blown managed services that manage Kubernetes control and data planes, as well as application lifecycle management, testing, deployment, monitoring and troubleshooting.

To get a better idea of the extent to which administration and operational activities are taken over by managed services, we will compare them with the operational breakdown from Kubernetes reviewed earlier.

Kubernetes breaks down the operational aspects of clusters into 4 distinct components: the data plane, control plane, cluster infrastructure, applications, and the range of cluster operations required.

The first set of managed services are Kubernetes cluster management tools. Cluster management tools and installers allow Kubernetes admins to create, upgrade, and operate Kubernetes clusters with some degree of high availability features built-in. With cluster management tools, developers have to manage all four operational aspects of Kubernetes clusters, which may result in higher management overhead and more extensive cluster operations management required. Some examples include Kops, Kublr, Minikube, Kubeadm, RKE and kubespray among others. Here is an extensive list.

Next on the list are turnkey cloud solutions. Turnkey solutions include Kubernetes offerings from multiple IAAS vendors such as AWS and GCE as well as providers like StackPoint, SUSE, Rancher, Giant Swarm and Containership.

Not all turnkey solutions provide the same feature sets. Most manage the Cluster infrastructure aspect of Kubernetes operations and build upon the cluster creation, upgrade and operational features of cluster management tools by providing built-in HA tools, monitoring, health checks and troubleshooting.

Since turnkey solutions take over some aspects of cluster operations, they require much lower management overhead and operational costs as compared to Kubernetes management tools. However, they still out-pace the operational costs of full-blown managed services. As with turnkey solutions, managed Kubernetes offerings also differ based on the range of operational activities that can be offloaded to them. Most managed services take over administration of the Kubernetes control plane, data plane and the cluster infrastructure.

This wide range of operational features means less management cost overhead and lead to more cost-effective Kubernetes environments. Popular managed Kubernetes offerings include Amazon EKS, Canonical, Containership, Google GKE, IBM Cloud Kubernetes Service, Microsoft AKS and OpenShift.

Managed Kubernetes offerings from cloud providers do have an inherent risk of vendor lock-in, but these risks are somewhat mitigated by the inherent portability of Kubernetes workloads. Most managed Kubernetes offerings from cloud providers also provide high availability, scalability and security features. Some offerings like Canonical and Openshift provide a more extensive feature set with support for deployment across cloud providers and on-premises, as well as application lifecycle management, CICD, deployment, testing, monitoring and troubleshooting.

Depending on workload size, infrastructure costs might contribute a bigger slice of total Kubernetes costs as compared to management overheads and operational costs. Day 0 Kubernetes planning also needs to take these costs into account. We have conducted a comprehensive investigation of these costs for major public cloud providers as well as openstack and on-premises. Our results indicate that past a certain infrastructure footprint on-premises installations are much more cost-effective as compared to cloud options. These results agree with investigations conducted by other independent parties.

Conclusion

As we have seen the operational costs of building Kubernetes from scratch far out-weigh the benefits accrued from the freedom and flexibility that come with it. Building Kubernetes from scratch and allocating precious developer resources to managing and operating it in-house might not be the best use of IT resources.

Managed Kubernetes services, which take over some of the heavy lifting of managing and operating Kubernetes clusters, are an attractive alternative for cost-efficient Kubernetes clusters, especially for SMEs and scale-ups.

Outside of niche, custom workload requirements, going with a subscription to a managed service provider will reduce the operational overheads and free up developer resources to work on projects that generate value and add to the bottom line.