Guest: Varun Talwar (LinkedIn)
Company: Tetrate (Twitter)
Show: Let’s Talk

Tetrate is driving new solutions as its influence in cloud-native security and AI infrastructure grows, highlighted by key partnerships including the U.S. Air Force and Bloomberg. In this episode, Varun Talwar, Co-Founder of Tetrate, discusses their work with the US Air Force on service mesh for Platform One, focusing on security and resource optimization through ambient mode. Talwar also covers the use of Open Security Controls Assessment Language (OSCAL) for continuous compliance and their partnership with Bloomberg to enhance Envoy with AI Gateway for handling LLM traffic and simplifying AI integration.

Introduction to Tetrate and its recent growth

• Talwar provides an introduction to Tetrate, talking about the company’s focus on building next-generation traffic management platforms using Envoy Gateway and Istio service mesh. Talwar highlights their offerings, which include both open-source and enterprise solutions aimed at modernizing service mesh infrastructures.
• Talwar mentions that Tetrate has experienced significant growth since their last interview, with a growing number of users and customers adopting their platform for secure, scalable cloud-native workloads.

Tetrate’s long-term collaboration with the US Air Force

• Talwar discusses Tetrate’s nearly collaboration with the US Air Force, which includes providing a service mesh for their enterprise DevSecOps platform. This platform is essential to the Air Force’s digital transformation and secure operations.
• Tetrate delivers essential container and runtime security, ensuring the safety and efficiency of the Air Force’s cloud-native infrastructure. The service mesh offloads security and networking tasks, simplifying the maintenance of secure operations.
• Talwar also highlights that Tetrate’s work has expanded over the years and now focuses on additional use cases, including improving continuous compliance to meet the rigorous standards required for national defense.

Introduction of ambient mode and new use cases in the US Air Force platform

• Talwar introduces ambient mode, a new feature Tetrate is integrating into the Air Force’s service mesh stack. This new mode provides Layer 4 (L4) capabilities without requiring a sidecar, marking a significant shift in how service mesh functions.
• Ambient mode’s key benefits include reducing resource consumption, lowering management overhead, and improving network latency. This makes the service mesh more efficient and easier to deploy at scale within the Air Force’s infrastructure.
• Talwar underscores that ambient mode allows for more flexibility in service mesh architecture, a crucial development for large organizations like the Air Force, which requires optimization across various platforms.
• Talwar touches on Tetrate’s advancement in container security and runtime security by introducing continuous compliance through the integration of the Open Security Controls Assessment Language (OSCAL).

Continuous compliance and integration of OSCAL for real-time monitoring

• The integration of Open Security Controls Assessment Language (OSCAL) enables Tetrate to provide continuous compliance monitoring for the Air Force. This system generates machine-readable reports for real-time assessments of compliance with standards like FedRAMP.
• Talwar highlights how this system transforms compliance from a static, yearly audit process into a dynamic, real-time function, aligning compliance with the constantly changing nature of cloud-native workloads.
• Talwar emphasizes that this step forward is not only impactful for the US Air Force but also for the broader industry, as it demonstrates how compliance can evolve to match the agility of cloud-native environments.

The importance of networking and security in the US Air Force partnership

• Service mesh technology, particularly Istio, shifts runtime security tasks from applications to the network level, allowing for centralized management of encryption, authentication, and authorization. This simplifies security for application developers.
• This centralized approach ensures consistent security across applications, enhances visibility for security and platform teams, and facilitates the implementation of policies like mutual TLS (mTLS), ultimately reducing the risk of vulnerabilities across systems.
• Talwar adds that this collaboration with the Air Force is setting a new standard for how security can be automated and made auditable, ensuring compliance can be continuously verified, a crucial step forward in both security and operational efficiency.

Partnership with Bloomberg to enhance AI infrastructure capabilities

• Talwar talks about Tetrate’s collaboration with Bloomberg centers around making Envoy and Envoy Gateway capable of handling large language model (LLM) traffic, which is increasingly being used in AI systems.
• The partnership aims to simplify AI integration by creating a unified API that allows developers to interact with different AI models, regardless of the underlying platform. This approach reduces complexity for developers working with multiple AI tools.
• Talwar highlights that these innovations include features like built-in authentication for OpenAI, AWS Bedrock, and other services, as well as rate limiting based on tokens (used in AI applications) instead of traditional request-based limits.

Development of the Envoy AI Gateway and industry-wide interest

• Talwar discusses the creation of the Envoy AI Gateway, a key result of the collaboration with Bloomberg. It will streamline AI integration by providing a single API that routes requests to different AI models, whether for handling HTTP, TCP, or LLM traffic.
• Talwar notes that this approach simplifies how AI models are integrated, while also addressing unique AI needs like authentication and token-based rate limiting. This makes the process of integrating AI into existing systems much smoother.
• The collaboration with Bloomberg came about after engineers from both companies identified similar needs for handling AI traffic, and they saw an opportunity to address these challenges collectively.
• Talwar mentions that since announcing the Envoy AI Gateway, other companies have expressed significant interest in adopting the solution, underscoring the growing need for efficient AI infrastructure across industries.

This summary was written by Emily Nicholls.