Guest: Gerred Dillon (LinkedIn, Twitter)
Company: Mycelial (Twitter)
Show: Let’s Talk

Mycelial is a new company that is building out a system for replicated data types at the edge. They are trying to serialize state over the wire and then re-hydrate it into objects to either side. This helps to empower people, from a distributed systems perspective, to use tools like conflict-free replicated data types (CRDTs), use web assembly and other edge-based technologies, without having to have a deep understanding of distributed systems.

According to Gerred Dillon, co-founder and CTO of Mycelial, after a good deal of research, what they discovered was “at the edge, sensors are generating an absolutely staggering amount of data. So IDC estimates that the data volume coming off of IoT devices alone will increase from 13 zettabytes in 2019 to roughly 80 zettabytes by 2025. And this is only going to grow more urgent, right?” Dillon saw an obvious problem in that existing network links cannot support this massive amount of data volume and realized, “As the compute power at the edge grows, everything from a watch to a temperature sensor, some of these can even run massive amounts of compute. They need to be able to make these processing decisions in a very low latency way.”

Dillon also mentions how crucial it is to Mycelial that data is portable over time. To that, he says, “as we start to stabilize, we’re open sourcing the protocols and everything that we’re doing enabling companies to have complete ownership of that, and not be just reliant on a software as a service there.”

The company recently received an influx of funding. With that funding, they plan to build out a system for deploying distributed actors. They are combining that with the best-in-class ideas coming out of academia like these CRDTs. Dillon explains this concept by saying, “The idea there is, if you and I are writing out to the same area, the system shouldn’t crash. It should offer people the ability to reconcile that information and move forward at a higher level than having to go solve that issue over and over again.”

The summary of the show is written by Jack Wallen

[expander_maker] 

Swapnil Bhartiya: Hi, this is your host, Swapnil Bhartiya, and welcome to TFiR: Let’s Talk. And today, we have with us Gerred Dillon, co-founder and CTO of Mycelial. Gerred, it’s great to have you on the show.

Gerred Dillon: Thanks for having me, Swap.

Swapnil Bhartiya: You folks raised 3.8 million to kind of securely keep data and state in sync for any app under any network condition in east three clicks. I want to talk a lot about the things that you folks are doing, but before that, since you’re also a co-founder, I want to know the story of the company and what is also the story behind the name?

Gerred Dillon: My co-founders and I were working in the cloud data space for the past five years. And as we started to see the cloud-native space mature, we often see that there was a missing component as more and more devices were pushing out to the edge. So you have a very mature environment from development to deployment to observability, but as soon as you head out to the edge, you end up in a chasm, and development there often looks like it did five to 10 years ago. So my co-founders and I were discussing this and realized there was an opportunity to really go solve these problems as softwares started to expand out to these edge areas, and we have more sensor data than ever.

Where we got to the name Mycelial was this idea that there is a mycelial network in the forest. And so if you think about the fungal kingdom, at the top you see mushrooms. But through the forest, you have an entire network of fungi that is supplying the forest. So mycelial network is able to transmit nutrients between trees and plants. You have instances where tree stumps are even kept alive for years by these networks. And then also, you have a whole bunch of other interesting things about them. So our name, it comes from this ability to efficiently federate and support entire ecosystems that rise above them.

Swapnil Bhartiya: Thanks for explaining that. Now, if I ask purely from a technical background perspective, what exactly do you folks do? You did talk about the challenges at the edge, but explain what do you folks do?

Gerred Dillon: So what we’re doing is we are building out a system for replicated data types at the edge, and this is a very difficult thing if you’re considering normal event-based systems. We go and we build out… You start a business, and event sourcing is incredibly popular. But what we’re ultimately doing is trying to serialize state over the wire and then re-hydrate it into objects to either side. So from a technical perspective, what Mycelial is doing is empowering people from a distributed systems perspective to use tools like conflict-free replicated data types, or CRDTs. We’re empowering them to use web assembly and other technologies at the edge without having to have a deep background in distributed systems to get that right.

Swapnil Bhartiya: You folks have been working in this space for a very long time, you and your founder. If I ask you, how has this space changed over the couple of years, because we are not only seeing new paradigms, we talk about DevOps, DevSecOps, all those new cultural paradigm. But we are also looking at a lot of technical shifts that are happening, a lot of folks are moving to the cloud. We are also seeing the emergence of edge as edge data center, not necessarily small tiny sensors. So talk about what kind of changes, trends, are you seeing in this space.

Gerred Dillon: Yes. So the first thing I want to talk about with that is an article written by Peter Levine back in 2017, called The End of Cloud Computing. And it was this observation that there’s an explosion of sensor data at the edge where we want to perform tasks in the real world around sensing, inferring and acting on things that are on data that is occurring out in the real world. And then from a centralized perspective, we want to learn about that and build better experiences. So there’s been this massive shift. We did a whole bunch of research as we were building this out and talking to customers and exploring. But what we discovered was that at the edge, sensors are generating an absolutely staggering amount of data. So IDC estimates that the data volume coming off of IoT devices alone will increase from 13 zettabytes in 2019 to roughly 80 zettabytes by 2025. And this is only going to grow more urgent, right?

So what we wanted to do was go in and respond to this because our network links cannot support this. We have more compute on either side of the network link than ever, but we cannot rely on a paradigm that relies entirely on making decisions at the cloud. So if you were to look at the past 10 years and the shift that’s going on is, if you look at the software as a service model, it succeeded because the cloud [inaudible 00:05:13] be able to be the complete arbiter of those decisions. And as the compute power at the edge grows, everything from a watch to a temperature sensor, some of these can even run massive amounts of compute. They need to be able to make these processing decisions in a very low latency way. So that’s the big shift we’re seeing is that in these environments, there’s often either not enough time, not enough bandwidth. There’s no way to maintain the quality of service across that link. And so there’s a bigger need than ever to actually do that local compute.

Swapnil Bhartiya: Excellent. I also want to understand how are things changing with data because the fact is apps can come and go. You can very easily… But data is where the real value lies. That’s the real asset of any company. When we talk about cloud, data gravity happens, a lot of… with vendor lock in, your data get locked. With Mycelial, how do you ensure that, or if you do ensure that users remain the owner of their data?

Gerred Dillon: The issues with SaaS companies and data is often, as you’ve just mentioned, SaaS companies come and go, the data is proprietary. And I do think in Kubernetes and cloud data cause a reaction that hurt SaaS companies in a lot of ways where that data became very proprietary, and created a demand for customers to own their own data. But the cloud data movement designed solutions to that in one way. So from a data ownership perspective, at Mycelial, we think that is incredibly critical that data is portable over time. So from one perspective, as we start to stabilize, we’re open sourcing the protocols and everything that we’re doing enabling companies to have an complete ownership of that, and not be just reliant on a software as a service there.

The other thing is ensuring portability. We have not designed systems where, for example, if I’m building an electric car and another company’s building an electric car charger, to be able to share and exchange data between those. And what we’re working on really heavily on Mycelial is to facilitate that ability for that exchange by unifying the concept of data with their compute. And I think this is really, really important as we’re generating more sensor data, as more data at the edge than ever, that not only is it portable, not only is its own, but it’s also open and accessible and usable, so that many entities can take advantage of it and work together.

Swapnil Bhartiya: Let’s now go back to the company and talk about the funding, talk about what are the areas that you folks are planning to grow? What are the areas you’ll be investing into?

Gerred Dillon: So the ideas we’re working on are really ones that are well founded over in a lot of spaces. So for example, we are built on top of something called the BEAM VM, Erlang [inaudible 00:08:24]. It’s well used in the telephony and telecom space. And one of the big principles of that was to create systems where individual components of that system can crash without affecting the rest of that system at all. So it enables highly resilient systems, and that is one of our core ideas of building very resilient applications and very resilient data in the face of a lot of real world issues, networks aren’t reliable, you may go unplug a device. It may just reach the end of its service life, right? If we’re talking about a satellite, it re-enters the atmosphere. But we still want to be able to access everything that we want to know about that and not have the entire network crash because something happens. And that’s one of our big ideas.

And so what we’re doing with that is building out a system for deploying out distributed actors, which is really great for building out digital twins. And alongside that, we’re combining that with best in class ideas coming out of academia, like these conflict-free replicated data types. And the idea there is, if you and I are writing out to the same area, the system shouldn’t crash. It should offer people the ability to reconcile that information and move forward at a higher level than having to go solve that issue over and over again.

If you look at event sourcing, which is fantastic, it often is bespoke in these various organizations where we start to solve a problem, hey, what do our events look like? Right? What the shape of the messages across the wire? And I think often that what’s really important in that, gets lost. So we want to be able to unify that instead of saying, “I’m going to go start a project. I’m going to create a product.” Instead of asking, “Well, what’s the shape of my event?” “Well, what’s the shape of my data?”, let’s focus on that. And then Mycelial’s the communications run time that then facilitates the most efficient way to move that across the wire. And so that’s how we use those ideas to actually perform that task.

One thing that we’re really noticing, and whether this makes it or not, with the chip shortage, we’re noticing that a lot more… you have the same product with often very different hardware profiles and makeups, and companies need to be able to deal with that if we’re not going to have just out of stock for months and months, and months, or years at a time. So that is one thing we’re really trying to facilitate is that, if we’re talking about abstraction layers and the ideas, that is one that’s very critical to us to improve the ability of teams and developers to deliver products despite those sort of challenges and constraints in the same way as a data tier.

Swapnil Bhartiya: Gerred, thank you so much for taking time out today and not only talk about the company, but also we talk about, of course, data and importance of ownership. Thanks for those insights. And I would love to have you back on the show, so I look forward to our next conversation. Thank you for today.

Gerred Dillon: Thank you. Have a great day.

[/expander_maker]