Open source communities are defining the next generation of infrastructure

0
Bio: Mark Collier is a co-founder of the OpenStack project and the Chief Operating Officer at the Open Infrastructure Foundation. A committed advocate for open-source principles, Mark has played a key role in the development and global success of OpenStack, establishing it as a leading cloud computing platform and one of the most active open source projects to ever exist.

A recently released Harvard study reported that the demand for open source has reached $8.8 trillion worldwide, a massive market opportunity.

Of course, there are hundreds of open source projects, but as COO of the OpenInfra Foundation, I can speak directly to the growth of OpenInfra software like OpenStack and Kata Containers and how that growth supports the assertions of the Harvard study.

Created in 2010, the OpenStack project has become the de facto open source cloud computing standard and one of the most active open source projects to ever exist in the world. Over the past 14 years, over 600,000 changes from 10,000 contributors have been incorporated into this project, and we foresee more growth to come in the coming years.

In fact, today we are seeing exponential growth and demand for OpenStack. Mordor Intelligence estimates that the OpenStack market alone will grow from $22.81 billion to $91.44 billion by 2029.

However, as we all know, the world is changing faster than ever. OpenStack has been successful over the past 14 years, but can it continue to be when the playing field is shifting so radically? We’re seeing new demands on infrastructure that haven’t been faced before, and we’re also witnessing some alarming trends that threaten the core philosophies of open source itself.

I believe there are four trends, in particular, that are already at work shaping the next generation of infrastructure. The open source community must respond to these trends and their inherent opportunities and threats in order to continue to contribute to the economy in a substantial way.

Trend #1: Digital Sovereignty

Digital sovereignty is about where your data is, who has access to it, and what laws apply to it. It has become a critical component of IT strategies. Organizations around the world are now incorporating digital sovereignty requirements into their planning and decisions about which technologies they choose to use.

Open source communities are providing the options that organizations need to meet digital sovereignty requirements. In France, for example, all of the largest banks in the country are running on OpenStack. Public cloud providers in Europe are choosing OpenStack for the same reason. There are over 300 OpenStack-powered public cloud data centers worldwide, and all of these are able to adapt to their geographical locations and the unique digital sovereignty requirements that they may have.

One of the largest European retailers operates a brand of grocery stores called Lidl. Last year Lidl had 1.9 billion euros in annual sales and it employs 7,500 staff. Lidl’s infrastructure runs on OpenStack, and all of their data is processed and stored within Germany and Austria. One of the big selling points of OpenStack is that the providers that are supporting Lidl’s public cloud are able to accommodate the stringent privacy and data protection laws of those two countries.

Another area where we’re seeing a lot of demand and change is hardware integration. The European Union’s technological sovereignty strategy centers around RISC-V, an open source hardware standard that is now being demanded by organizations around the world. The OpenStack community is integrating with this standard, and the two open source communities are working together to deliver software and hardware that works and meets the digital sovereignty needs of our brave new world.

Trend #2: Licensing Changes

Another trend that is defining open source today is licensing changes. It’s been well documented in the media these last few years how several companies have broken the trust of open source, moving away from open source licenses to proprietary licenses — decisions that are bad for customers and the market.

The open source response has been overwhelming. In some cases, open source communities have launched new open source alternatives. OpenTofu is a great example of how the open source community stepped up to provide an open source alternative to Terraform when HashiCorp switched to a proprietary license.

More recently, the media coverage of this disturbing trend of licensing changes has been dominated by Broadcom and VMware. VMware was acquired by Broadcom, and subsequent licensing changes made organizations worldwide reevaluate their virtualization strategy, and with good reason: some organizations were faced with a 10 times increase in the cost of platforms they already have in production.

In this case, the OpenStack ecosystem has stepped up and offered OpenStack solutions as a viable alternative to VMware virtualization.

We did a poll of our OpenInfra Foundation members, and over 80% of them have already talked to someone about migrating workloads from VMware to OpenStack, and 60% of them have already completed a migration. Responding to the VMware licensing change is going to be a defining moment for OpenStack and will boost market growth for the open source project as organizations turn to it for executing their cloud native infrastructure strategies.

GEICO is a case in point. One of the largest auto insurance insurers in the United States, GEICO has $32 billion in assets and over 40,000 employees. Geico has successfully migrated workloads from VMware to OpenStack. And GEICO is just the beginning: we expect hundreds more organizations to make this change and grow the OpenStack market share.

Trend #3: Security

Everyone needs security within their infrastructure, so it’s something the open source community is very committed to delivering.

One of the interesting trends with respect to security is that almost every organization has started to adopt containers, or they’ve had them in production for several years. But here’s a frightening reality: It is estimated that as much as 87% of container images in production have critical or high severity vulnerabilities. No one wants that within their production infrastructure.

So, at the OpenInfra Foundation, we’ve been really excited to support the Kata Containers project. Kata Containers is an open source container runtime that offers the security of VMs with the speed of containers. The efficacy of this project has led to many organizations worldwide securing their infrastructure with Kata Containers. Examples include Microsoft, which is using it with their Azure product, as well as NVIDIA, who’s securing GPUs with Kata Containers.

A recent addition to this list of Kata Containers users is AWS. AWS is starting to run Kata Containers in production and has published some excellent documentation for customers who want to enable Kata support for their workloads.

Kata Containers is yet another example of how open source communities are responding to trends, identifying needs and rising to the challenge, delivering software that ensures that infrastructure providers have what they need to meet the demands and challenges now and in the future.

Trend #4: AI

AI is completely redefining cloud infrastructure, both from the software and the hardware perspectives, due to the unique demands that it places on data centers. According to Blackstone Group, datacenterHawk has documented an 11-times increase in data center demand between 2019 and 2023. And Edge Delta reports that more data has been created in the last three years than in all of history combined!

One might be tempted to call AI the “elephant in the room” because of the huge impact it is having, but the idiom doesn’t apply because we’re not avoiding the topic—rather everyone is talking about it! In fact, most organizations are planning to put AI to use and are already thinking about how to adapt their infrastructure to accommodate AI workloads. According to one recent report, 96% of organizations polled plan to expand their AI compute capacity.

Jensen Huang, the CEO of NVIDIA, is predicting that the installed base of data centers, which already has a $1 trillion market value, is going to double to $2 trillion, and all of these will be using accelerated compute. A lot of this AI-driven, exponential growth is going to be around providing the storage and networking that is required to accommodate AI workloads. Data centers, from a software and from a hardware perspective, simply must adapt in order to get the job done.

Open source communities are already hard at work responding to these AI demands. The UK’s fastest supercomputer, which is called the Dawn supercomputer, is already running AI workloads based on OpenStack in production and the engineering team expects to scale considerably. And NexGen, which is one of the largest buyers of GPUs, also says that OpenStack is the backbone of their infrastructure. As you can see, organizations are already relying on OpenStack — and its excellent track record for supporting accelerated compute workloads in high performance computing (HPC) — to support AI workloads. They trust OpenStack not only for what it can already do, but also for what they know it will grow to do.

That trust is well placed. The OpenStack upstream community is responding to AI needs in every single release. OpenStack is never standing still. It’s the software community that never sleeps. The OpenStack community releases new versions every six months. The most recent release, Caracal, is from this past April 2024, and one of the key features that was delivered was the vGPU live migration, a critical feature in supporting AI workloads.

How will open infrastructure communities rise to the challenges of these trends and new trends yet to emerge? The trends I’ve just mentioned are not insignificant. The challenges are substantial. That’s why open source communities are uniquely qualified to address them. It will take all of us working together. We need users like the Dawn supercomputer and NexGen identifying needs and gaps; we need leading hardware players like Nvidia, AMD and Intel working with the development community to ensure hardware compatibility; we need corporate and individual contributors delivering upstream features; and we need other open source groups collaborating on standards and common solutions. Together we can make full-stack open AI infrastructure a reality.

We’ve done it before; we’ll do it again.

As mentioned above, open source has a trillion dollar infrastructure market opportunity — and I consider it a responsibility — to drive the growth of the digital economy.

The last trillion dollar opportunity was in telecommunications. Ten years ago, the largest mobile carriers in the world came to our international summit and told our community they needed open source network function virtualization to prepare for 4G and 5G. We delivered. Now nine out of the 10 largest telcos in the world run OpenStack.

We’re going to do it again for this next generation of infrastructure by doing the three things we do best: bringing everybody together, identifying the gaps, and working to manage the upstream development process.

We’d love to have your help. The great thing about open source is that we always welcome new contributors, new users and new ecosystem members. If you would like to play a part in defining the next generation of infrastructure, please join us.

Importance of determining the right level of protection | SIOS Technology

Previous article

Akamai simplifies and accelerates API discovery and protection with its new connector | Stas Neyman

Next article