Author: Mark Lavi, Principal, Cloud Native Product Manager, Veeam Software
Bio: Mark manages Kasten and Veeam’s open source efforts for Kubernetes data protection and application disaster recovery, security, and migration. He aligns upstream, downstream, and Kasten project, engineering, and working group efforts across many open source and business partner communities.

In Veeam‘s recently released 2024 Veeam Data Protection Trends Report, more than half of the organizations surveyed have Kubernetes in production! Those who protect cloud native data are equally split across four administration team categories: Kubernetes, Database, Storage, and Backup (see page 8: Containers are Everywhere, but Their Backups are Scattered).

Persistent workloads on Kubernetes are growing, particularly databases. The latter are complex, distributed across cluster nodes, and despite replication methods, their data protection requirements exceed storage snapshots. Application consistency is a requirement for database backup and disaster recovery, cyber-resilience against ransomware, and audits. The DevOps divide between teams, applications tiers, and their operations often challenge or block persistent workloads moving to Kubernetes. From the developer perspective, the ability to create, test, and ship improvements is often painful, isolated from cloud native, divided across teams, and slow when incorporating a database. Without a mandate to drive GitOps skills and outcomes into an organization, the necessary team partnerships and responsibilities needed will block agile databases.

Enter Kanister.io, one of the newest CNCF (Cloud Native Computing Foundation) sandbox projects, a workflow for persistent workload management. Kanister Blueprints make data operations (DataOps) easy by orchestrating logical tooling, Kubernetes DB operators, and external concerns for application consistent backup and restore. There are two benefits to this approach:

1. Analogous to infrastructure as code, Kubernetes plus Kanister make operations as code possible for databases. Bringing database operations to software developers unlocks high performing technology teams.
2. Disaster recovery is achievable. This business goal is overlooked by developers and added to the application by other teams after development. By uniting DataOps with developers, the strategy for disaster recovery and RTO (Recovery Time Objective) becomes an extension to the design.

Overcoming the delegation of data to operations, storage, backup, and database administration teams, which may not be skilled in cloud-native operations, unlocks the potential of data in software development and testing. The challenge of treating databases and other persistent workloads as an atomic entity made up of the runtime, data, and operations has been achieved: databases are agile!

Adding databases and their data operations to Continuous Integration and Delivery (CI/CD) pipelines and software testing becomes easy. Much like the golden images for operating systems, application VMs (virtual machines), and base container images, a snapshot database can be manipulated for development, testing, and production environments with better performance than synthetic creation from scratch. With proper vetting, derivative production data and production load testing is possible, allowing the promotion of data from production to development to testing to staging environments. With Kanister, agile databases with data can be included in application integration testing and deployment.

Kanister blueprints are YAML and Kanister is a cloud native controller, easily installed by Helm chart onto a Kubernetes cluster, accessible via CLI (Command Line Interface) and API, and ideal to integrate into pipelines.

Bringing operations as code into pipelines is a logical step, but day two operational maintenance is kept separate. The next step organizations take is to use GitOps. GitOps as the source of truth for application configuration and automated operations drives organizations to higher levels of performance and quality. Many organizations limit GitOps to stateless workloads, such as a stateless web tier or micro service, because data operations with persistent workloads is too hard, fragmented, or considered unachievable. GitOps plus Kanister blueprints enables full application control to include the database and other persistent workloads. Best of all, the combination enables resource efficient controls for storage, CPU, and network that wasn’t easily achievable in a cloud native manner before.

Want to learn more?

• Visit https://kanister.io to adopt the project and find application blueprints
• https://opengitops.dev for an overview of GitOps principles
• Veeam is a Silver member of the CNCF, a Gold Sponsor of KubeCon + CloudNativeCon Europe in Paris from March 19 – 22, and the original Kanister.io contributor. There are multiple conference learning opportunities:
  • Stop by Veeam’s booth (E11) for in-person Kanister demonstrations
  • The Kanister.io booth in the CNCF Project Pavilion is open on Wednesday from 10:45 am to 3:00 pm
  • The Data on Kubernetes Community Day co-located event on Tuesday to discuss persistent workload challenges and developments

To learn more about Kubernetes and the cloud native ecosystem, join us at KubeCon + CloudNativeCon Europe in Paris from March 19-22.