With Snowflake’s growing adoption as a cloud data platform, organizations are generating and storing data at an unprecedented scale. But here’s the catch: finding the right data in Snowflake can feel like searching for a needle in a haystack. Without a proper system to organize and surface relevant datasets, analysts waste hours hunting for tables, engineers struggle to understand dependencies, and governance teams face mounting challenges in ensuring compliance.
This is where a data catalog comes in. A robust data catalog is no longer just a "nice-to-have"—it's an essential tool for any organization using Snowflake at scale. Read on to explore how organizations approach centralizing their Snowflake metadata into a data catalog – some choose to implement their own solution building on Snowflake’s native capabilities while others like our customers Pitney Bowes and nib adopt automated solutions.
Table of Contents
- Snowflake Horizon Catalog: Built-In Governance and Discovery
- Snowflake Open Catalog: Centralized Catalog for Apache Iceberg Ecosystems
- Select Star: End-to-End Data Catalog for Snowflake and the rest of your data stack
- The Path Forward with Data Catalogs
Data catalogs serve as a bridge between internal metadata and the needs of data consumers. Within Snowflake there are 3 types of data catalogs: Snowflake Horizon Catalog, Snowflake Open Catalog, and Select Star’s Data Catalog.
Snowflake Horizon Catalog: Built-In Governance and Discovery
Built into the Snowsight UI, Snowflake Horizon Catalog is a governance and discovery solution that Snowflake announced in 2024. Snowflake Horizon Catalog enhances data management by providing features such as object tagging, lineage tracking, AI-powered search, and granular access controls, allowing users to more efficiently find, understand, and trust data within the Snowflake environment.
Snowflake Horizon Catalog is designed for data analysts, engineers, and analytics teams that regularly interact with Snowflake, particularly those focused on streamlining query management and job orchestration.
What’s missing with Snowflake Horizon Catalog:
- Missing user-driven collaboration, like annotations, custom documentation, or team communication features that facilitate knowledge-sharing.
- It lacks integrated business glossary to define, manage, and standardize business terms and metrics, limiting consistent business context across teams.
- Limited cross-system integrations for metadata and lineage tracking with external data stack tools such as Tableau, Looker, Power BI, dbt, Fivetran, or Airflow.
Snowflake Open Catalog: Centralized Catalog for Apache Iceberg Ecosystems
Snowflake Open Catalog is a managed service for the interoperable open-source catalog Apache Polaris. Open Catalog provides a unified interface for secure, cross-engine access to Iceberg tables while maintaining atomic metadata operations and role-based security controls. By decoupling catalog management from storage layers, it enables organizations to enforce consistent access policies across Spark, Snowflake, Trino, and other Iceberg-compatible engines without duplicating data or permissions. The service integrates natively with Snowflake's ecosystem while supporting hybrid cloud storage configurations on AWS S3, Azure Blob Storage, and Google Cloud Platform. With features like granular role-base access control, catalog mirroring, and managed infrastructure, Open Catalog addresses critical interoperability challenges in open lakehouse architectures while maintaining Snowflake's signature ease of administration through Snowsight UI integrations.
Snowflake Open Catalog primarily serves data teams managing multi-engine environments that require vendor-neutral catalog capabilities. It is particularly valuable for organizations adopting open table formats like Iceberg, as it eliminates siloed metadata management and reduces operational overhead associated with maintaining custom catalog solutions.
What’s missing with Snowflake Open Catalog:
- Functions more as an information schema rather than a comprehensive data discovery and governance tool for end users.
- Its focus on Iceberg translates to a lack of support for non-Iceberg table formats as well as other types of data assets.
Select Star: End-to-End Data Catalog for Snowflake and the rest of your data stack
While Snowflake offers several approaches to data catalogs, these approaches all lack the advanced features of a robust cataloging solution like detailed data lineage or a business glossary. In contrast, automated data catalog solutions like Select Star simplify Snowflake metadata management while extending its value across your entire data stack.
Select Star is designed to serve as your definitive source of truth for data. Select Star carefully balances meeting the technical demands of your data team and ensuring accessibility for business users. This dual-purpose design makes Select Star an ideal choice for organizations seeking to bridge the gap between technical and business-oriented metadata management needs.
- Advanced Capabilities:
- AI-Powered Documentation: Automatically generate table/column descriptions and answer natural language queries about your data.
- Cross-Platform Lineage Tracking: Visualize upstream/downstream dependencies across Snowflake and external tools like BI dashboards.
- Autogenerated Entity Relationship Diagrams: Discover what tables and fields to join for a query without depending on tribal knowledge.
- Usage Tracking: Get visibility into which datasets are actually necessary for your analytics.
- Cost Optimization Insights: Attribute compute spend to specific tables or users to identify opportunities for cost savings.
- Faster ROI: Active metadata within 24 hours and save hundreds of hours on avoiding maintenance and automating repetitive tasks like dataset tagging or usage monitoring.
- Scalability & Security: Purpose-built infrastructure designed to handle enterprise-scale metadata securely.
Read more about how to connect Select Star to Snowflake and then try it out in a free trial.
How Nib Transformed Data Discovery with Select Star
Nib, a major health insurance provider in Australia and New Zealand, implemented Select Star alongside their Snowflake environment to overcome challenges in data accessibility and governance. With over 1,500 tables across multiple environments, their data team struggled to find and understand the right data. This led to inefficiencies, with analysts spending up to 30% of their time searching for information and verifying its accuracy. The lack of clear data lineage and documentation also posed governance risks.
To address these issues, nib implemented Select Star as an automated data catalog solution alongside Snowflake. This strategic move dramatically improved their data discovery and governance processes.
- Centralized metadata allowed teams to find relevant and trusted datasets instantly.
- Automated documentation reduced manual effort by over 50%.
- Improved governance workflows ensured compliance with regulatory requirements.
- Analysts could now quickly locate and trust the data they needed, reducing search time and increasing confidence in their analyses.
As a result, Nib achieved faster decision-making processes while saving significant engineering hours.
How an Automated Data Catalog Saves Pitney Bowes $1+M and Hundreds of Hours Every Year
Pitney Bowes, a global shipping and mailing company, struggled with data discovery and documentation across their complex data environment. With over 25,000 tables, their data engineers spent excessive time answering basic questions about data assets. By adopting Select Star as their automated data catalog, Pitney Bowes significantly improved their data operations. The solution automatically documented data assets, provided clear lineage, and offered powerful search capabilities.
- Easily identified stale objects and enabled Pitney Bowes to increase Snowflake workload by over 50% without affecting costs
- Saved over 30 hours per month on data-related queries, allowing their data team to focus on more strategic initiatives.
This transformation highlights how an effective data catalog can streamline operations and enhance data-driven decision-making in large enterprises.
The Path Forward with Data Catalogs
As data environments continue to grow in complexity, the need for effective data cataloging becomes increasingly critical. While an internally developed catalog may address near-term needs, most organizations will benefit from dedicated, automated data catalogs to help them find and understand their data in Snowflake and other platforms. By investing in robust data catalog solutions, organizations can transform their approach to data discovery, governance, and utilization, ultimately driving better business outcomes through more efficient and informed decision-making processes.
If you’re considering a data catalog to help users at your organization find the right data and accelerate self-service analytics, we’d love to help. Find time with our team to explore how we can help.
