With companies rushing to move to the cloud, many find they are presented with unexpected costs. As a result, companies are now starting to look more closely at their cloud spending and optimize costs. In this video, Tom Nats, Director of Customer Solutions at Starburst, sits down with Swapnil Bhartiya to discuss the topic of the month (T3M), cost optimization. On discussing companies jumping in feet first to the cloud, Nats says, “I think people just get so fast into it, and then it’s too late, like we spent way too much money.”

Cost optimization for data lakes with Starburst

• Nats discusses how many companies are focusing on cost optimization nowadays, not just for applications but also for data lakes. Audit companies are becoming more common at events like AWS:reinvent aiming to help companies save money. Costs in the cloud can quickly mount up.
• Nats explains the differences between cost-cutting and cost optimization for data lakes and how new technologies like Apache Iceberg can help organizations cut costs by querying only the necessary data.
• The rush to the cloud has led to a lack of consideration for cost management, resulting in unnecessary expenses.

Cloud cost optimization and approval processes

• Nats highlights some of the unexpected costs of cloud computing, such as a $390,000 bill for an application that spits out a single word (“null”) every second for four years.
• Sometimes customers have just accepted a bill thinking that is how much they pay for storage without thinking about the usage and whether it is necessary.
• Nats discusses the challenges of managing cloud costs, including unlimited resources and lack of approval processes, and suggests implementing approval processes and tools to mitigate these issues.
• Nats tells us that organizations are looking at moving some workloads to third-party data centers to help manage cloud costs.

Cloud computing costs and optimization

• Nats reluctantly says that many companies are going back to the old-school way of approvals. He believes that we need to see more barriers or gates in place to avoid costs getting out of hand.
• Cloud can provide cost savings and improved resource allocation but it can be a challenge for Starburst Enterprise customers to shift their mindset from on-prem to cloud.
• Nats provides an example of a company using a cloud-based solution to split costs and gain visibility into report usage, leading to cost savings and more efficient resource allocation.
• Multi-cloud environments can actually lead to increased costs due to egress costs and nowadays this can look like a lock-in. Starburst Stargate helps optimize costs by only sending the data that is needed.

Cloud cost optimization and generative AI adoption

• Nats discusses cross-cloud cost optimization and the trend of edge computing. He talks about the considerations of pulling data directly from the edge rather than dragging it back to a central repository.
• Nats shares a personal experience from their past job at a sports company, where they used a flash report to quickly access sales data without the need for extensive ETL or MicroStrategy processes.
• Generative AI is a double-edged sword with the potential to be used to reduce costs for companies, however, the cost of hiring data scientists for this purpose is considerably expensive.
• Nats feels skeptical about the value of expensive data scientists, whose work does not always lead to an ROI. It is beneficial to do some experimentation before jumping in.

Guest: Tom Nats (LinkedIn)
Company: Starburst (Twitter)
Show: T3M

This summary was written by Emily Nicholls.