Edgeless Systems has announced the launch of Continuum, a security solution that enables cloud-based “Confidential AI” services for the first time. At its core, Continuum ensures that user requests (or “prompts”) and the corresponding replies remain encrypted throughout within an AI service, making plaintext data inaccessible to both the AI service provider and the underlying cloud infrastructure.

Furthermore, it protects the often highly valuable weights of AI models in the same manner. To achieve this, Continuum leverages the confidential computing capabilities of Nvidia H100 GPUs in concert with cutting-edge cryptography and innovative sandboxing technology, allowing AI service providers to freely update models and code without compromising security.

Continuum works with large language models (LLMs) like Mistral 7B and supports popular AI serving frameworks like Nvidia Triton Inference Server and vLLM.

Felix Schuster, CEO of Edgeless Systems, said, “Continuum will significantly influence the generative AI market, allowing a wider range of enterprises, particularly those in regulated sectors, to confidently utilize offerings like chatbots. This product embodies our mission to enable the use of cloud services without compromising on data security or compliance.”