Validate, optimize and manage Snowflake Gen2 with Slingshot

Databricks and Snowflake offer incredible capabilities that enable organizations to manage, process and analyze vast amounts of data. But, as many platform leaders know, scale introduces complexity. The sheer number of pipelines and queries, combined with the variety of teams accessing these platforms, makes it challenging to maintain a clear view of utilization and spend across the organization.

Capital One Slingshot was designed to provide you with this visibility and control. Our goal is to free data and platform teams to focus on the things that matter most by making it easier to manage data infrastructure at scale.

Today, we’re excited to share a few new features that double down on this mission. From lowering the risk of testing Snowflake Gen2 warehouses to streamlining day-to-day management of your data clouds, Slingshot makes it easier to discover, understand, resolve and validate data cloud operational issues.

Let’s dive right in.

Snowflake Gen2 warehouses are the next generation of the virtual warehouse. They leverage faster hardware and intelligent software with enhancements to table scans and delete, update and merge (DML) operations. As a result, they tend to provide better performance than Gen1 warehouses, especially for DML‑intensive and scan‑heavy workloads. 

Indeed, the performance implications seem promising. But, they come with a catch. Gen2 warehouses consume 25-35% more credits per hour than Gen1 counterparts. So, how do you know what workloads to test Gen2 on with minimal risk? That’s where Slingshot comes in.

Slingshot provides a low-risk way to test and validate whether Snowflake Gen2 warehouses are right for your workloads: 

• Test with confidence: You can create a generation 2 standard warehouse or convert a Gen1 warehouse and monitor cost and performance impact using our dashboards and alerts.

• Optimize with a schedule: Once you’ve validated Gen2 is right for you, Slingshot helps you optimize those warehouses with a dynamic schedule. We recommend hourly schedules based on historical performance and provide estimated savings and runtime impact, so you can make informed decisions. 

With Slingshot, you can confidently adopt the latest technology where it makes sense, and stick to Gen1 where it doesn't.

Compute optimization is critical, but so is the day-to-day management of your data clouds. We learned this firsthand at Capital One as we scaled our own usage to thousands of users running millions of queries a day. 

When you are operating at that scale, manual tasks become bottlenecks. We’ve taken what we learned from our own journey and built those best practices directly into Slingshot.

Here are the new features designed to streamline your operations:

• Bulk tagging for better cost allocation: Slingshot tags enable both cost attribution reporting and access controls. You can now bulk tag objects across multiple environments. This solves the headache of tagging data objects one-by-one and creates a scalable foundation for unified cost allocation and federated governance.

• Actionable alerts where you work: Email isn't always the right channel for time-sensitive issues. Slingshot now integrates directly with Slack, Microsoft Teams and PagerDuty. This enables your team to respond faster to cost spikes or performance anomalies. By meeting your developers where they work, Slingshot reduces the time it takes to detect and respond to operational issues.

• Context with Snowflake query tags sync: You can now sync Snowflake query tags directly into Slingshot. Query tags allow you to attach metadata to every query execution. By syncing these into Slingshot, we help teams understand how changes to queries impact workload costs and performance.

• Weekly warehouse recommendations: Workloads change constantly. Slingshot now provides weekly optimization recommendations for Snowflake warehouses, helping you adapt to changes faster than ever before. 

Sometimes, the hardest part of optimization is knowing where to start. You know there is waste in the system, but finding it across thousands of tables and jobs feels like searching for a needle in a haystack.

That’s why we built free health checks dashboards for both Databricks and Snowflake: 

The Databricks health check: This is a dashboard you can import directly into your workspace. It leverages metadata from System Tables to surface insights across Jobs, APC, SQL warehouses and Lakeflow Declarative Pipelines.

• Identify your most expensive jobs, users and resources

• Find overprovisioned and underprovisioned resources

• Detect slowly growing workloads that might pose a future risk

• And more

The Snowflake health check: Available on the marketplace. This app queries ACCOUNT_USAGE and ORGANIZATION_USAGE tables to identify inefficiencies and opportunities for optimization across warehouses, storage and queries.

• See when you will run out of credits

• Find zombie storage and duplicate tables

• Identify your most expensive warehouses and tables

• And more 

These health checks can help guide your optimization efforts with immediate, actionable lists of things to look into or fix.

At Capital One Software, we are committed to building tools that help data and platform teams manage and optimize their data infrastructure at scale. 

This starts with compute optimization but extends far beyond that to the day-to-day management of systems and pipelines.

• Slingshot’s Snowflake Gen2 warehouse optimization feature makes it easy to test and validate this new compute. See where it should be applied to lower costs or improve performance, then use our optimization recommendation to optimize Gen2 warehouses on an ongoing basis.

• Tagging, notification and health checks are the newest set of features designed to help data and platform teams manage their data clouds and focus on high impact optimizations rather than administration.  

The goal is simple: We want to give you the confidence to scale, the visibility to control costs and the freedom to focus on the data, not the infrastructure.

If you’d like to learn more about how Slingshot can streamline data management for your organization, book time with the team here.