Swapnil Bhartiya: Hi, this is your host Swapnil Bhartiya and today we have with us Mike Fuller, Member of the Technical Staff, Tokenomics Foundation. Mike, it’s great to have you on the show.

Mike Fuller: Thanks for having me on.

Swapnil Bhartiya: It’s my pleasure. The Linux foundation recently announced the intent to form the Tokenomics Foundation. First of all, talk a bit about that foundation.

Mike Fuller: Yeah, so over the last sort of six months, definitely in the last four months, we really saw a need for an open space for the conversation to be had around the AI value and how organizations are approaching their AI practices. The FinOps foundation, which I’m part of, has always looked at the technology value stack, but we realize that there’s dimensions on this conversation that go beyond what finops traditionally had been covering. And so to give it a dedicated space within the Linux foundation for that conversation to be had and to invite key members that you know that are using or being part of the delivery of AI features in order to help all organizations globally around the use of tokens.

Swapnil Bhartiya: The foundation talks about tokens as the core unit of the AI economy. I’m heavy user of AI. I think 99% of the stuff that I do these days is through AI, whether it’s my workshop or of course creative work. And token is what actually we sweat. Oh my God, so much. Tokens are going to cost us a lot. Can you talk about how are you helping organizations connect token usage to bigger measures like efficiency, ROI and overall value? Because sometimes what happens is that these things don’t connect very well.

Mike Fuller: Yeah, so I think similar to the journey that we saw in cloud, just much faster and I think a little bit more confusing because it’s not technology stack that we’re quite familiar familiar with in the past. So first and foremost we really need to start to help people understand not just what is a token, but what is all that infrastructure that goes around that token. And then once you understand the visibility of the cost that’s being charged, there is then starting to look at the cost per outcome or the tokens per outcome within the is stack that people are using. So there’s sort of two dimensions that we’re seeing companies look at. There’s the internal AI or the productivity AI and then the product AI. So the, the AI used for, for organizations product, you know, services and product that the customer facing and so just trying to figure out like a good way to measure and get your arms around that spend and then to actually associate it to the outcomes that the business is seeing from the use of the AI itself.

Swapnil Bhartiya: When it comes to AI you know, it it AI workloads, they bring together, of course, computer data and model behavior all at once. How is Tokenomics foundation approaching standardizing these pieces so teams can actually understand cost and performance instead of looking at many different things?

Mike Fuller: Yeah, I think that’s one of the things that I think we need to solve in a lot of engineers heads is the AI stack or especially when you get to gentic harnesses and stuff like that, it kind of looks like an amorphous blob of technology cost coming out of this AI inference layer. And we’re trying to sort of build that architectural diagram inside of everyone’s heads, understanding exactly what, what components are in that architectural diagram and which ones cost what, what, you know, how we can think about the actual drivers of cost within that architecture and then allow teams to, you know, understand why all of the choices they’re making within that, that technology stack. So you know, it’s the token itself is something that it sort of boils it down to something that’s fairly simple because you can just put a dollar figure to a token, especially when you’re using it from the clouds or the frontier model providers. But it gets more complicated as you start to do a mixture of technology stacks and routing different prompts to different locations about what that is actually costing the business and how that associates to the outcome.

Swapnil Bhartiya: Based on your interactions with community organizations, how much are people worried about token token cost?

Mike Fuller: Yeah, so I think, you know, we saw state of finops data over the last couple of years move from most FinOps teams thinking about AI cost to now, pretty much every one of them managing AI cost in some way, shape or form. At a recent conference in San Diego we had a lot of a focus on AI and it was a mixture of large parts of the community going yes, we’ve needed this, it’s right where we are. And then it was funny because we had individual interactions with a few practitioners that were like, oh, I’m not sure about, you know, if this is really a thing yet. And then we had phone calls from them a week later after the conference saying that I got back to my desk and my CTO come flying down and I’m all about trying to manage our AI spend now. And so I do think that it’s either right on the the cusp for most organizations to start thinking about the AI value conversation or it’s already a

Swapnil Bhartiya: high level concern today when it comes to token cost. And I may be totally wrong. It, I think is applicable only when we are using APIs. But if we are running models locally, then token cost doesn’t matter. It only depends on the context window and other things. So when we talk about token cost, it’s mostly about API consumption, is that correct?

Mike Fuller: No. So I think when we look at tokens that are generated locally, we do start from, you know, simple components that go into the ingredients that go into that. So your, your energy, your cooling, your, your space for the equipment. There’s the whole procurement cycle of, of equipment and the lifecycle management of the equipment itself and then the, the actual architectures of that hardware and the models and engines that you choose to run on it. All of that effectively goes into the cost of having inference, which you know, is the token at the end of it in order for you to gain value from AI. So it’s basically a, that’s one, a good example of area that was kind of beyond what a traditional finops practitioner was looking at. It’s sort of going back to some of those old routes of capacity management and hardware acquisition in the data center. And it is all driven around getting to a point of having an efficient token generation within the data center.

Swapnil Bhartiya: There are some effort, if I’m not wrong, like anthropic cloud and of course the fable came or other things, the new version since I use heavily, so I am. They try to optimize it, but it directly affected the quality as well. So the thing is, if you try to tame the token consumption cost, it may directly affect what is the long term solution technologically, socially or you know, just culturally.

Mike Fuller: Yeah, I think that that is one big difference. When we look at optimizations from, you know, where we’ve looked at them in the past, where usually you’ll have a fairly sort of fixed static understanding of the workload capacity needs and it’s really just fitting the hardware to that. In this case, some of the optimizations actually affect the quality of the, of the work that’s being done and is trying to figure out good practices around balancing those. We effectively have our own series of parameters at the same time moving up and down, if you will. On the optimization side, where does it go? You know, I think it’s going to be an improvement in tooling potentially using AI itself to help us make the choices. You know, definitely see that there’s a lot of opportunity for some experimentation and best practice development in that area.

Swapnil Bhartiya: If I look at the foundation, you folks have broken tokenomics into three major buckets. Production, consumption, and of course monetization. From technical standpoint, where do you see the biggest opportunity Right now is there to improve efficiency or visibility?

Mike Fuller: Yeah, I think consumption is probably the one where the conversation goes to naturally especially if you are using a lot of the frontier models via an API like you say, so that the tokens generated outside of your, your area but it there, you know, it’s trying to figure out exactly for each org, you know, where those opportunities really lay. And I think consumption is going to become a more of an interesting one for the wider industry as we continue to see the open weight models, you know, improving in quality. The choices of, you know, a mix between the frontier models using hosted open open weight models or running open models yourself will become part of the convers with organizations to have. And we don’t want to swing the pendulum too far the wrong way and drop the quality of AI and drive up the amount of equipment we have to now manage just to do the AI inference. There’s an opportunity cost balance there. And then on the value side of things, you know, the volatility, I guess of the token price and the amount of token consumption being unpredictable into the future really will impact the sort of value that you’re getting and the monetization of the tokens, especially when they’re customer facing tokens. And so businesses do have a lot of conversation to think about how they’re going to package those prices and costs into their products suite.

Swapnil Bhartiya: And as the foundation builds out, of course open frameworks and standards, can you talk about what are your top priorities for making sure that everything stays of course interoperable and clear across different models and platform? Because everybody is mixing models, they’re using different platforms.

Mike Fuller: Yeah, I think just, just the same as you know, we saw with, with finops when it comes to the different cloud providers and them having different terminology and different structures, there are some underlying similarities of course. So it’s going to be a mixture of finding the common language and trying to build that language across the way we talk about AI and AI infrastructure and then specializing into, you know, leaning in where there’s particular terms or particular types of activities that are specific to you know, one or two vendors. And so that there’s a base level of common understanding and then particularly, you know, hot areas being covered specifically where they are unique in particular pockets when

Swapnil Bhartiya: I’m talking to you, of course the cost, the whole finops, you know, it started when we started to tame cloud cost and there are some clear parallels. Now the difference is that cloud itself cannot solve or fix the problem of how cost, cloud cost or egress ingress fee, AI can help solve some of its token cost problems. Can you talk a bit about what kind of telemetry or measurement capabilities do teams need to really manage AI cost and outcomes and what lessons, if any, we have learned from taming cloud cost?

Mike Fuller: Yeah, so I think the cloud bill and working with a detailed billing file is definitely a skill that’s going to come into high value here. But what we are seeing with the AI is we can’t just love the whole activity of AI into a cloud bill or you know, like structure, you know, especially when it comes to the open specification, we have like focus because it will grow, it will just explode the granularity of that billing data. And so I think it is one area where we are going to have to learn to be quite, become quite fluent in pairing up a billing data set with your hotel observability data set. So you’re really looking at, you know, do we need to have down to session or down to individual operation in the cloud build? Probably not, but we should have some telemetry around that because we can’t really just say, hey, we’ve spent this number of tokens per hour over the last month. We need to be starting to break that down to these are the teams that are consuming them, these are the applications that are using them. You know, these are the particular types of expensive operations we’re doing that are enabling us to actually get an understanding of the cost and the cost opportunity that’s there.

Swapnil Bhartiya: I mean, it’s not that somebody is getting started, but a lot of organizations, they’re already in the middle of their, you know, AI journey and it gets so exciting that they, they totally forget about the token cost and it’s only when they get the bills then they realize it. What are the first practical step you would recommend to these organizations? Somebody who is getting started, let’s say, towards measuring token economics inside their own AI stack so they can control it before it goes out of control. Yeah.

Mike Fuller: So I think what we’re seeing with the sort of more advanced practices that are on the leading edge of this curve is first they’re trying to decide exactly how much of the AI highly trying to tackle. And so they will look at, you know, is it the internal AI, is it the product AI? You know, are we trying to tackle both at once or are we going to start with one area and then expand out? So I think the, you know, trying to tackle key areas of the AI spend and not trying to grab it all at once is probably the first recommendation. You know, the productivity AI is One that seems to get a lot of tension because of it’s where teams are using it in agent flows and you know, with those having retry loops and parallel prompting and all sorts of things that can drive that token consumption up. And so it’s trying to find the, the amount of surface area that you look at driving for that visibility. So the mixture of putting in observability metrics on the, on that use and also looking at what billing you have available. Unfortunately, we are, you know, in a world kind of where we were right at the beginning of cloud, where the billing data is quite nascent and not, not detailed enough. And so there’s a, you know, for us there’s a concerted effort to try and get practitioners to push for better billing data from the model providers and cloud providers and the frontier model providers in order to get that granularity we need in order, in order to get the visibility and understanding of the cost that’s there. So some, some early steps would be just deciding how much of this pie you want to buy and then trying to push for better telemetry and billing data.

Swapnil Bhartiya: One thing that we have learned from AI is not ask how things will look like five years from now. If you can tell me how things will look like five days from now, that would be that great. But if you look at the foundation, what would success look like for Tokenomics foundation? Where you’ll see this is what we wanted to do and this is what we have achieved.

Mike Fuller: I think first and foremost we reduce the amount of uncertainty that organizations have when they look at their AI spend. I think that we’ve seen that transition when we look at the cloud spend journey that we started out where most organizations were worried about where the cloud bill was going, they weren’t sure if they had control of it. They were worried that it would outstrip their revenue growth. I think today we feel most practitioners talk about their cost forecast to be around single digit, low, single digit forecast. We need to be in that sort of area in the next, you know, a few years, hopefully less. It’s going to move so quickly where we’re feeling confident that our forecasts on AI spend are, you know, close to what we end up with and businesses know, you know, where they’re putting their investment dollar on AI. So it is a, it’s more of a confidence generation for success for the Tokenomics Foundation. Can we get to a point where businesses feel confident?

Swapnil Bhartiya: What are the best practices you would recommend for folks so they don’t compromise on how they use AI they don’t compromise on output they get, but they can still contain the token cost.

Mike Fuller: Yeah, I think, you know, when it comes to very large organizations with deeper pockets, I think it’s quite common that they will throw money at the wall with their innovation. You know, I think it’s two things. It’s one, trying to find the next business differentiator or it’s B, trying to make sure that their workforce is as productive as they possibly could busy. So it’s, you know, spend the tokens to get there really good. I think for most organizations, however, there will be some level of governance that comes into upfront where you’re setting some expectations of spend on tokens out with your organization. You’re seeing more and more cost control levers being implemented in the frontier models and on the cloud platforms as far as AI token consumption use. So I think it’s going to be just a smart level of guardrails being put into place for most organizations who don’t have hundreds of millions of dollars to spend on experimentation and then looking at identifying where those tokens being spent is actually having a good business impact and where they’re not and trying to reshuffle, you know, those investments as early as possible. Because if you want the sort of the most outcome, you want to be putting the investment on the right token. I think the main thing though is, is that for all organizations is they’re going to have to be comfortable with some token spend not having a return that it’s part of this learning exercise. We saw it, you know, with any technology where you start, figure out exactly how it has value and then to slowly learn where to invest better in a technology stack. And I think AI is just this on accelerated timelines.

Swapnil Bhartiya: Mike, thank you so much for joining us and sharing how the Tokenomics foundation is going to tackle this problem. Thanks for your time and as usual, I look forward to chat with you again. Thank you.

Mike Fuller: Thanks. It’s great to be on your show. I think for us it’s just making it clear that we don’t have all the answers. We’ve created a space specifically for us to explore and work through all of the questions, especially those that you’ve given us today and continue to refine the answers to be right on point and correct and develop those best practice frameworks to give companies better guidance in this space.

Swapnil Bhartiya: Excellent, thank you. For those who are watching, please go and check out Tokenopics foundation and since this is all open source, please also get involved. Mike, once again, thank you.