Author: Markus Eisele
Bio: Markus Eisele is a Red Hat Developer Tools Lead at Red Hat. He is also a Java Champion, former Java EE Expert Group member, founder of German JavaLand and a speaker at Java conferences around the world.

If there is a driving force in software development today, it’s scaling the most critical part of the enterprise: the developers. For the last several years we’ve been building up, and enjoying, CI/CD (continuous integration and continuous delivery) systems like ArgoCD and Tekton that work to deliver software faster. Now architects are expanding their vision to Platform Engineering in order to combine entire suites of tools under a common developer experience.

A key part of this synthesis is observability. Easy access to information and holistic views of systems reduces human error and also increases efficiency, speed and standardization. Current open source efforts underway are uniting dashboards, environmental sustainability, and AI together through developer portals to give developers a clear and encompassing view when they need it.

Coding for Sustainability

In the past, developers’ main concerns for their code were reliability and performance. Today, code concerns are much broader, encompassing security, regulatory requirements and even environmental and energy costs of running the code. Teams need to be able to quickly and easily–even automatically–be aware of their applications’ “carbon cost.”

“At Project Kepler, we’ve expanded developers’ toolkits to address the crucial environmental impact of running microservices– the code of the cloud native era. Our goal is to seamlessly integrate Kepler’s power consumption calculation capabilities into the application lifecycle management, enabling teams to efficiently and effectively evaluate and eventually optimize their environmental impact footprint,” said Huamin Chen, co-founder of Project Kepler.

Kepler is providing increasingly sophisticated energy cost metrics, integrated into CI/CD pipelines. Kepler has included Code Carbon approaches, where every code object is analyzed for impact on energy consumption, on its roadmap. By exposing this information, development teams can both optimize for more efficient energy use and avoid costly mistakes that greatly increase waste.

Dashboards as Code

The DevOps movement introduced us to the concept of “infrastructure as code,” which has grown and blossomed into GitOps. The adoption of Kubernetes-based platforms enables teams

to manage both platform and applications through the same Git-based continuous integration process. Today, that includes the dashboards that provide system visibility.

At Promcon 2023, Prometheus maintainer and Amadeus principal software engineer Augustin Husson introduced the Perses Project. Since Prometheus is the leading datastore for Kubernetes stack monitoring, Perses enables building dashboards as code to expose its data. This lets developers and operations staff collaboratively maintain their own views of both applications and platforms.

“At Amadeus, we were able to create a wide range of well-maintained dashboards by designing them ‘as code.’” explained Gabriel de Labachelerie, Head of Frontline Services, Amadeus. “These dashboards are essential in managing our critical production systems. I believe that Perses, with its ‘as code’ first approach, will improve observability and production of operations across the entire IT industry.”

As a best practice in platform engineering, Perses is built around a plugin infrastructure. Thus it becomes a platform for building dashboards more than a dashboard application.

Building Smarter Tools

Today’s engineering platforms must provide the tools developers need, not just to write code, but to build platforms and microservices that integrate with complete platforms.

For example, AI developers must integrate approved and monitored models. A software catalog of all available components, including deep learning models, can reach into available model types for generation or predictive analytics. A complete platform will not stop there but it will also enable developers to use smaller models locally, almost as a digital twin, to seamlessly integrate with the full-blown production models at a later stage.

Advanced tooling also helps make sense of the vast (and often confusing) galaxy of microservices and components that makes up a modern Kubernetes-based stack. The Korrel8r project monitors signals in a Kubernetes environment and uses these to chart dependencies and interactions between objects. This exposes relationships in the application infrastructure to developers, making it easier for them to understand the impact of changes and how to make improvements.

“Korrel8r automatically finds related data in the large, mixed bag of observability signals generated by a cluster,” explained Alan Conway, Korrel8r Maintainer. “This allows administrators and developers to navigate quickly from the symptoms of a problem to all of the diverse signal data that is related to the symptoms. This includes logs, metrics, traces, events and other data flows. Korrel8r’s flexible design allows it to relate different types of signal data even if the signals were not designed to be correlated. This is important in the

current world of mixed signals, ranging from unstructured log records (the least standardized but most common signal) to modern, standardized trace spans.”

Tying it All Together through Portals

Making effective use of all of these transparency tools requires systems that are intuitive, efficient and enjoyable for developers. Integrated developer portals (IDPs), often based on the Backstage project and can adapt to existing tools and processes, both preventing a fractured tooling landscape and delivering a central place for all relevant project information and observability. IDPs enable the building of tailored environments with the tools and data that developers need, when they need them.

This experience goes beyond a single landing page with bookmarks for easy access. The focus is shifting from just building and deploying software to making these processes more intuitive, efficient and enjoyable for developers. The push for a better developer experience will see more intuitive interfaces, streamlined workflows, and deeper integrations between tools. Examples include tools like Podman Desktop that allow developers to work locally with containers, but also push them into test environments and work with Kubernetes configurations in the same way this is done in staging and production environments. Even deeper integration can be achieved with integrated open source projects like Janus IDP, for example.

“As developers continue to prioritize simplification and speed, the role of integrated developer portals and platform engineering are changing the way developers work. Embracing them is going to be instrumental for high performing teams in the future,” predicted Markus Eisele, Developer Strategist at Red Hat.

The role of observability and platform engineering in shaping the developer landscape becomes increasingly crucial. Advancements in practices such as GitOps, requirements including environmental sustainability, and technologies such as AI and machine learning promise transformative shifts in the way we develop and deploy software. It’s clear that platform engineering will continue to play a pivotal role in driving innovation and efficiency for cloud-native developers.

###

To learn more about Kubernetes and the cloud native ecosystem, join us at KubeCon + CloudNativeCon Europe in Paris from March 19-22.