Author: Dean Steadman, Manager, NetApp Astra Product Management
Bio: Dean Steadman is a product manager with more than 25 years of IT experience. He focuses on designing, deploying, protecting, supporting, and breaking hybrid cloud environments. Dean has accumulated hands-on knowledge at companies of all sizes, from cool internet start-ups to stuffy Fortune 500s. He enjoys gaming, hiking, and successful system upgrades.

By now you’ve read eight million blogs claiming that “Kubernetes is the de facto container orchestration platform,” and I’m happy to jump on that SEO train. It’s impossible to argue with the scalability, portability, and flexibility of Kubernetes when it comes to bringing applications to our end users. However, with great power comes great responsibility. And with great speed and flexibility comes the opportunity to fail spectacularly. You don’t have to go far in the headlines to find horror stories. Whether it’s an attack, a simple mistake, or events beyond our control, every data protection team carries the weight of the world on their shoulders.

The requirements that are placed on data protection teams have, for the most part, remained consistent. Applications need to be online and accessible by our users. That means that the underlying data must be secure, replicated, and protected. And, of course, it all has to be done within an established budget. Cheers.

Kubernetes simplifies the deployment and scaling of containerized applications, but it also introduces complexities regarding data management and protection. Traditional methods of data backup and recovery don’t always fit seamlessly into the dynamic and ephemeral nature of containerized environments. Kubernetes applications often span multiple clusters, nodes, and pods, making it challenging to maintain data consistency and integrity. Simply put, it’s a lot for backup teams to wrap their heads around.

Let’s start with the challenges that are associated with data protection in Kubernetes.

Resilience. Kubernetes workloads are designed to be highly available and resilient. However, achieving the same level of resilience for the underlying data can be complex. Failures can occur at various levels, from nodes to pods, and it’s crucial to ensure data availability during these events.

Consistency. Maintaining data consistency across multiple replicas and pods is essential. Data protection solutions need to guarantee that all copies of data are synchronized and up to date. Automation is the key to maintaining this critical timing and should be baked into solutions at every step.

Recovery. In the event of data corruption or loss, rapid and reliable data recovery is critical. Because of the distributed nature of many applications, traditional backup and recovery methods may not work seamlessly in a Kubernetes environment. Admins need tools that know the complex relationship between containers and their persistent volumes to ensure that all the pieces are put back together.

Compliance and security. Ensuring that sensitive data complies with regulatory requirements and maintaining data security are paramount. The speed and frequency of updates to Kubernetes environments can expose data to new security risks, and data protection solutions must adapt accordingly.

Fortunately, many of the solutions to these challenges come from established best practices, common sense, and a new generation of tools that keep pace with the speed of Kubernetes. Here are just a few of the ways to overcome these challenges.

Start with 3-2-1. They say that the classics never die, and I have to agree with them here. The old adage of having three copies of your data, stored on two different media types, and with one off-site copy holds true today for Kubernetes. Three copies of our data is relatively easy when we have the combination of production data, snapshots, and backup images. I use a loose interpretation of two media types because the cloud obfuscates many of our media selection options (does Amazon even offer tape?). The use of two storage protocols or solutions is close enough here. For example, using a file storage solution for production while using object storage for backups fulfills the spirit of the practice. And finally, having a replica available in another pod or region completes our basic data strategy.

Embrace the new. Just like organizations are increasingly relying on Kubernetes to orchestrate their containerized workloads, backup teams need to embrace it to protect those workloads. Data protection solutions must understand the dynamic and ephemeral scaling nature of applications and must scale alongside the applications. To ensure backup consistency and application recoverability, solutions also need to embrace the breadth of storage options and access modes. Kubernetes presents unique challenges that may not be completely covered by legacy data protection solutions. To promote the smoothest experience, teams should consider tools that are designed specifically to protect Kubernetes.

Test, analyze, optimize, repeat. Our end users, our companies, and our employment status depend on our ability to protect data and to recover it if failures occur. Having a great strategy and deploying the best tools are an excellent starting point, but they’re useless until they’re proven to work. Michael Jordan tells us, “Practice like you’ve never won.”

Procedures must be frequently tested to ensure that the team knows how everything works and that applications can be fully restored. Each test needs to be reviewed and analyzed not only to ensure that the test itself is valid, but also to identify improvements that can be made. Optimizations kick off a new round of testing and validation, and the cycle hums along. The secret to a great night’s sleep is knowing that your data is protected.

We’re just starting to leverage the benefits and impacts that containerization and Kubernetes have on IT workloads. These benefits are spreading beyond the DevOps teams who initially embraced them to speed up and to scale application delivery. Many infrastructure and data protection teams are embracing Kubernetes and DevOps practices to orchestrate solutions to become more efficient. However, the underlying responsibilities and tenets of our roles remain the same: Keep the lights on and keep the business running.

NETAPP, the NETAPP logo, and the marks listed at https://www.netapp.com/TM are trademarks of NetApp, Inc. Other company and product names may be trademarks of their respective owners.

Join us at KubeCon + CloudNativeCon North America this November 6 – 9 in Chicago for more on Kubernetes and the cloud-native ecosystem.