Salt Labs was created in 2021 to help the industry with tackling the increase in API threats. The research division of Salt Security focuses on not only finding API vulnerabilities, but also increasing awareness about API security and offering solutions to help mitigate such risks.
We hosted Michael Isbitski, Technical Evangelist at Salt Security, to learn more about Salt Labs, the changes in API security landscape, their latest report around GraphQL authorization flaws and commentary on NSA’s Kubernetes hardening guide.
We started to talk about security in the cloud-native world only recently. While there is awareness about security in the space, there is still a huge gap between the idea of security and actual implementation. “There’s definitely a lot more awareness around API security issues, but we are actually seeing a lot of the same mistakes, unfortunately,” says Isbitski.
GraphQL vulnerability is a good example of what he sees as a pattern now. “We saw those same things with GraphQL in a recent threat research report,” he says. Those mistakes tie back to authentication and authorization or are related to business logic and exposing too much data. Salt Labs have found that those mistakes are pretty common.
On that, Isbitski adds, “It’s not so much that we’re seeing injection-style attacks. Those are still pervasive, certainly in the application space with cross-site scripting or SQL injection.” Isbitski continues, “A lot of the API attacks much more frequently target the business logic or authentication and authorization mechanisms.”
With the latest NSA announcement, Salt Labs is helping by raising awareness with content and other types of materials. Isbitski explains, “We’ve created sets of best practices. We’ve created evaluation tools.” With Salt Labs, it’s also about continuously educating the public and businesses. To do this, Isbitski indicates that it’s about asking the right questions. What are the nuances? Why did this happen? What were the flaws? Do they map to the OWASP top 10, or was it something more? What are the potential business impacts?
As to the awareness of security practices, Isbitski believes part of the problem is that “there hasn’t been standardization on how componentry is reported. So there’s discussion around what an SBOM (Software Bill of Materials) should contain?” Isbitski continues, “There hasn’t been universal adoption around these things yet. I do think it’s going to come back because of the Cybersecurity Executive Order that the Biden administration put forth.” Of course, SBOMs can often point to problems with dependencies, which Isbitski concludes, “We’re on the right path, but there needs to be an expanded definition of, ‘What is a dependency?’.”
About Salt Labs: Salt Security protects the APIs that form the core of every modern application. Its API Protection Platform is the industry’s first patented solution to prevent the next generation of API attacks, using machine learning and AI to automatically and continuously identify and protect APIs. Deployed in minutes, the Salt Security platform learns the granular behavior of a company’s APIs and requires no agents, no configuration, and no customization to pinpoint and block API attackers.
About Michael Isbitski: Michael Isbitski is a Technical Evangelist at Salt Security, helping to improve awareness and technical understanding in the area of API security. Prior to joining Salt Security, Michael was a Senior Director Analyst at Gartner for Technical Professionals (GTP) within the Security Technology and Infrastructure team. He researched and advised on a range of application security and infrastructure security topics including API security, security testing, secure design, secure SDLC, application protection, container security, Kubernetes security and secure continuous delivery. He has guided hundreds of organizations of all sizes globally in their security initiatives, across sectors and verticals. Additionally, Michael has over 20 years of hands-on practitioner and leadership experience in the fields of application security, vulnerability management, risk assessment, enterprise architecture, and systems engineering.
The summary of the show is written by Jack Wallen
Swapnil Bhartiya: Hi. This is your host, Swapnil Bhartiya, and welcome to TFiR Insights. And today we have with us, Michael Isbitski, technical evangelist at Salt Security. And we are going to talk about a new availability that was discovered for a GraphQL API. But before we go there, Michael, first of all, it’s great to have you on the show. We have covered Salt on a regular basis, but could you tell us, what does Salt do?
Michael Isbitski: Yeah. Thanks, Swapnil. It’s great to be here. So, Salt Labs is actually our threat research division. Salt Security actually creates an API protection platform, provides full life cycle detection and protection capabilities for all types of APIs.
Salt Labs, specifically, does threat research on all things APIs. Launched it, I think, around July timeframe. It’s been a few threat research reports that we’ve published. And most recently we had the GraphQL authorization flaws, which we’re going to be talking about today. But Salt Labs, like any kind of threat research division, they’re actively looking for known types of weaknesses. It’s substantially different than something like [inaudible 00:01:14] recently. But known vulnerabilities that end up with common vulnerabilities and exposures, CBE IDs in this world, we’re kind of looking for kind of unknowns, right? You might say there’s zero days, but it’s misconfigurations, misimplementations, design flaws, poor coding practices, right? It can be any number of things that lead to an attackable API or abusable API.
Swapnil Bhartiya: Excellent. Thanks for explaining what the Research Lab does at Salt Security. If I ask, from your perspective, especially when we look at the whole track or attack service in the cloud space, it has been changing, we have I think we started talking about security like two or three years ago. [Inaudible 00:02:05]you’ll see a lot of, sessions were there or security. So it does appear that the community is taking security seriously, they’re talking about at all about it, but there a difference in having aware it, talking about it versus implementing it in practice. So what have you seen especially even we do look at the whole lab that was set up there at Salt.
Michael Isbitski: It’s a very good question. And actually, prior to joining Salt Security as a technical evangelist, I was a former industry analyst with [Gartner 00:02:37]. Prior to that, security practitioner. So I’ve seen quite a bit in my time. Getting back question though, there’s definitely a lot more awareness around API security issues, but we are actually seeing a lot of the same mistakes, unfortunately. We’ll probably talk a little bit about it today, but [inaudible 00:02:56] web application security project, they have a project dedicated to API security issues. They’ve graded the top 10 for APIs 2019, and it’s actually held very accurate. So we see a lot of those same issues.
We actually see, we saw those same things in this GraphQL on a recent threat research report, but they’re seemingly simple mistakes, kind of tie back to authentication and authorization quite frequently, or related to business logic or exposing too much data. But the mistakes are pretty common, right? It’s not so much that we’re seeing injection style attacks, right? Those are still pervasive, certainly in the application space with cross-site scripting or SQL injection. And yeah, I mentioned [inaudible 00:03:51], right? That’s specifically injection type flaw, but actually a lot of the API attacks much more frequently target the business logic or authentication and authorization mechanisms.
Swapnil Bhartiya: Since we briefly touched upon awareness, NSA also came up with [inaudible 00:04:07] which is all about API. So there is a lot of awareness going on there. What does it mean for Salt security? Because as I said, detecting threat is one part of the problem, ] helping users is second part of the problem. And third of the problem is to actually bring it in their clutch itself. We do talk about DevSecOps and SREs and everything else, but so can you talk about these three stages?
Michael Isbitski: Yeah. So a lot of what I do at Salt is raising awareness, right? So it’s a lot of what I’m a little bit of the public face of the company, you might say with the content that we create and awareness materials, thread research reports that certainly help kind of talk about these publicly. We’ve created sets of best practices. We’ve created evaluation tools. If you are looking at specific tooling to help with API security. So I think part of your [inaudible 00:05:01] it’s bigger than just selecting tools. So, how are we helping on that front? I mean it’s continuously educating, right? When there is a public incident, some of what I do is actually kind of acknowledge that, right? Well, we had a problem, it’s really not helpful to say it’s poor coding practice. Quickly dismiss it, [inaudible 00:05:21] in the details, right?
What are the nuances? Why did this happen? So I try to do a little bit of investigation kind of dig in, what were the flaws? Do they map to the OWASP top 10? Or was it something more? What are potential business impacts? That’s something that’s often missing when you read about security incidents, maybe it’s too much technical detail, not enough of kind of the business context, right? And that’s important as a practitioner, right? You have so many fires you’re dealing with, you can’t address them all. You have to prioritize things. So what is the potential business impact to your organization? And then what are the things you can do to actually work on the problem? I love to say, “Purchase the Salt Security API protection platform.” But not everybody necessarily has the budget yet for that, but there are some things you can do to kind of mitigate right, raise the bar for attackers, make that a little bit more difficult for them to exploit or review your APIs.
Swapnil Bhartiya: It’s not a good thing, but we have a use case here, which we can discuss to not only get deeper into how, of course [inaudible 00:06:24] but also how you help a customer details. You folks recently uncovered the GraphQL API authorization vulnerability. So this will actually help us understand a lot of things. So first of all, tell us what it is, how effects organization who are using graph TL, because that’s what you talk about. Right. We have to assess the impact on the business. So let’s start there.
Michael Isbitski: Yeah. And I mean, we could take a few steps back, right? Why are our organizations even adopting graph QL as opposed to rest APIs? We’re seeing it because there’s benefits to front-end design, typically that’s performance or functionality, right. You can create kind of more responsive user interfaces. Typically that’s a mobile app, right, which is kind of why Facebook created the technology originally and then open sourced it. But we’re seeing a lot of adoption of GraphQL just because it can help vastly improve user experience as opposed to rest API calls, which you might say are too chatty. Right. We have to invoke too many of them to fetch a piece of data or provide a piece of functionality to that application front-end. So, I mean, that’s the business case for GraphQL. The problem with GraphQL is it’s kind of, wouldn’t say a design flaw, because it’s actually by design, but GraphQL allows you to nest multiple queries, mutate queries, right?
So you, as end user, you submit one request to that GraphQL endpoint. And then in the back-end, there’s GraphQL server. And then that fires off any other number of API calls to complete the function, right, provide you the piece of data you need. So there’s a lot of actually added complexity that starts to come into the fold when organizations adopt GraphQL. Right. And it’s not just GraphQL , they are still keeping rest as well. So you have this combination of protocols and API design types. So as you can imagine, right, it kind of creates some complexity. If you think about, if I’m able to nest multiple API calls and one GraphQL call, that can get a little hairy, right, because depending on which piece of data I’m fetching, do I know that that end user is authorized to that piece of data or functionality, right?
And if you think of even a small handful of APIs nested within that GraphQL call, it can be a problem, right? We’ve actually seen even issues with Facebook where they’ve had issues with their GraphQL API not properly authorizing API callers. So those kind of authorization flaws are kind of inherent, right? There’s much higher likelihood that the API designers are going to make those mistakes, which is exactly what we saw in this particular piece of threat research, which was with a FinTech platform, but they had architected their GraphQL API, which was worked in tandem with the mobile app front-end. And then users would kind of invoke things that way.
Lo and behold, the GraphQL API was not properly verifying authorization levels of that user [inaudible 00:09:32] two different pieces of data. So in the OS language that’s referred to as Broken Object Level Authorization, but plain and simple, it’s an authorization flaw, the API caller, what they are authenticated for in their session, you are entitled to certain pieces of data. But what happens with attackers is they can frequently manipulate API calls, parameters in URLs, message bodies, right? It can be different pieces of that API call that they tamper with and then they can attain access to other data or functionality.
Swapnil Bhartiya: You touched upon it briefly. But if I ask, as you said earlier that customers can go and buy Salt Security, but not everybody can do that or, so what are the things that users can do right now to kind of protect themselves so that they are kind of relatively protected from this authorization vulnerability that is there?
Michael Isbitski: I would say most conversations around API security actually start with inventory, right? A lot of our customers, they come to us at first because they acknowledge that they’re having a really hard time discovering all of the APIs they have in their enterprise. Even in this case of GraphQL, it’s one GraphQL API, but you’re talking tens or hundreds of back-end or [inaudible 00:10:52] APIs or rest APIs, you might call them inner APIs. Right. It’s much more than what most enterprises would think of. Typically they think of their portfolio in terms of applications, maybe infrastructure, right? What servers power, all of those applications, but very rarely do they think of it in the context of this application is actually built of these hundred APIs. Right. And things are much more integrated than that too.
So, one app likely intersects with 10 other applications particularly when you’re working with partners and suppliers. So, APIs are fundamentally the thing you should be looking at because that’s what attackers are targeting. So, inventory is a big one. There are things you can kind of pull in, static data sources. Most organizations are working with API gateways or API management today. You have an API catalog there, hopefully, right? Hopefully you’re formally publishing APIs in some way as part of your API mediation strategy, but that is part of your API catalog. The other way you kind of augment that is low balancers, delivery controllers. Maybe you’re running forms of vulnerability assessment. You’ll be capturing API calls as part of that, typically the hard part of that kind of manual approach, right?
This is in the absence of tooling because how you asked me, right. But this is the problem, right? How do you stitch together all that data? How do you know the API context? Like what are the right parameters for given API? What’s the metadata that belongs to that API? That’s where our organizations typically run into problems. And then API discovery is actually their big business driver for seeking API security tooling. But if the API inventory is a known quantity, it’s making sure that access control policies are set correctly. If you are using something like [inaudible 00:12:53] revisit scopes, are they set appropriately? Are they granular enough? Or are they two core screens, right? This can happen very frequently when you talk about that balance of outer and inner API calls with consumers that are external to your organization.
You’re not dealing with employees within the enterprise where you’re dealing with a very large mobile consumer base. And then other approaches that we often see that don’t always work, rate limits. Every gateway is going to provide you a form of rate limiting. Most organizations have challenges scaling them. So essentially you need a way to kind of track what are the normal consumption patterns, like how should this API function and that front-end working in tandem with the back-end APIs? That’s really where you need kind of higher levels of capabilities with data analytics.
What is the normal baseline? When you detect a deviation, that’s typically where you’d want to adjust the rate limiting, or maybe challenge for additional factors of authentication. It can vary depending on how your API is being abused, but it’s much more precise. We’re not just blocking entire IP address spaces, which historically doesn’t work. Right. And API needs to be available. Users need to be able to use the app. So you have normal consumption and then you have attackers abusing the APIs. It can be very hard to discern, which is normal and which is abusive.
Swapnil Bhartiya: Right. And you were talking about inventory. We also start talking a lot about SBOMs, [Software develop materials 00:14:30] even [inaudible 00:14:31] just around that as well. That was mentioned there. Once again, since you work in a [inaudible 00:14:38] phase of the company to educate awareness, how much awareness do you… You also mentioned your companies do want to know, how much awareness is there about SBOMs? Bill of materials that, there are a lot of open source SPDXs. There and so many other tools out there, but again, the basic point we are back to awareness. How much is there and what companies should do?
Michael Isbitski: Yeah. Great question Swapnil. A topic that was near dear to my heart, as an analyst, I did cover kind of open source componentry and how do you get a whole of that, SBOMs can be a way, I mean, the SBOM is the software build materials, like knowing what are all the things that go into all of your applications, but also infrastructure, right? You mentioned [Kubernetes 00:15:23] it’s built up of components and then it has an API. It’s declarative infrastructure. Very big problem for a lot of organizations, they were definitely asking those questions around how do I get a handle? And I’d say it kind of started with, I believe it was Equifax, right? When they had, I think it [inaudible 00:15:39] and then they were breached as a result of that.
So that allowed our organizations to start to look at, how do I know? What is all the open source componentry and use? We’re seeing this again, coincidentally with Log4j because it is a very similar problem. But SBOMs weren’t really solved. There is plentiful tooling, but there hasn’t been standardization on how componentry is reported. So there’s there’s discussion around what should an SBOM contain? Not so much formats. You did mention FedEx and [inaudible 00:16:16] is another one. But there hasn’t been like universal adoption around these things yet. I do think it’s going to come back. Right? Because the Cybersecurity Executive order that the Biden administration put forth, I think it was like may timeframe talked about some of these concepts within digital supply chains, because like this is another problem. When you start thinking about where your source componentry from or the third parties you’re working from? What are all the pieces of software they’re using?
This is where SBOMs come back into the picture. So some of that language is in there. So I think we’ll see some movement on this front. The problem, there aren’t enough problems with SBOMs other than that data standardization is that it’s too exclusively looking at software as opposed to APIs because the way a lot of enterprise application development occurs is, we’re building web APIs, web applications mobile apps, and things are stitched together using web APIs. So web APIs haven’t become part of that conversation of SBOMs yet that I’ve really seen, which is unfortunate. But at some point I think we should start seeing that because that is the supply chain in addition to open source component. So with that plus more. So we’re on the right path, but there needs to kind of be expanded definition of, what is a dependency?
Swapnil Bhartiya: Right. When you talk about [inaudible 01:10:40], I think in September SPDX became an ISO standard. It was accepted there. So there are a lot of progress that is happening there. So SPDX company has standard does help, but yes, there are different problems. But the thing is when you look at Cloud Native, things are so complicated. There are so many [inaudible 00:17:57]. You don’t even know where to look. So that’s where it does help sometime, the best solution is to go with a vendor, right. Because then you don’t have to do with that. So let’s luckily talk about Salt Security is really. How you folks help companies, because it’s overwhelming, where to look. All you want is to run a business, but you get stuck in so many different things that business becomes the least on the priority list. So please talk about that.
Michael Isbitski: Yeah. No, it’s also very a very good question. So and you brought up a good point too, right? Infrastructure is code, which is yet another format, right? Yamo, XML you name it, depends on what’s the technology stack you’re working in. There’s a lot of things that can be done, I’d say on the left side of the equation. Right. And this is, you could group it into shift left and there’s many things we could do as security to kind of try to address things early. That’s where a lot of these conversations around, what does the codified thing look like? Then can we audit that predelivery to kind of avoid security issues that could be exploitable or abusable? Kind of it’s a security amplifier in a sense. There’s a few problems. One is it’s very hard to audit all types of code, particularly when you talk about app code versus infrastructure, which is the reality, right?
As part of Cloud Native design, you have to be looking at everything because it is that combination of application and infrastructure that powers apps and APIs. The reality is there’s always a difference between the intended design and the documented design and then what’s actually being built and delivered. So at Salt, we stress, certainly look at the left side of the equation, audit your API schema definitions. We have those capabilities, find things that are, you might say low hanging fruit or maybe it’s a simple misconfiguration. There’s challenges with specs though and I talk about this when I speak at other events. There’s only so much you have to define in a schema definition, right. It might not even be present. We actually see that pretty commonly in a lot of organizations or maybe is only there at the launch of an API.
And then things start to drift, right. Or documentation process falls off. So it becomes stale. So production is always going to look a little bit different than what you’ve your intended design. Then it gets worse when you think about partners and suppliers, like what are they doing? Are they giving you the right things? So, certainly look at the things on the left side to help kind of improve your API security program. But you also have to be looking at the runtime side of the equation because that’s that’s reality. That’s your production traffic. Can we compare that to the discrepancies that are occurring in build?
As part of design with schema definitions absolutely. Right. And help you identify that, that’s part of our security insights that we can help development and engineering teams to kind of close that loop. Let’s fix it kind of improve the security posture of our or APIs overall. But it is an ongoing process. It’s kind of that combination of left, right. And then it is a continuous loop, much like DevOps itself.
Swapnil Bhartiya: Michael, thank you so much for taking time out today and talking about, of course not only GraphQL authorizing vulnerability, but also the much larger problem that we are facing in this space. And what you folks at Salt Security are doing to not only help us identify those problems, but also solve them. So thanks for those insights. And I would love to have you back on the show. Thank you.
Michael Isbitski: Awesome. Thanks Swapnil. It’s really big. Good to be here. Happy to help anytime. Thank you.