Modernizing data governance processes is no longer optional—it's imperative.
The Gartner Data & Analytics Summit shed light on the pressing need for businesses to adapt and innovate their data governance strategies to stay ahead of the curve.
Mona Rakibe, co-founder and CEO at Telmai, and Shinji Kim, founder and CEO at Select Star, shared their hot takes from this year's summit, a gathering for chief data analytics officers (CDAOs) and data and analytics (D&A) leaders aspiring to transform their organizations through data, analytics, and AI.
Data Governance Needs to Adapt to GenAI
The integration of GenAI applications into daily operations introduces new complexities and challenges for data governance.
A keynote session highlighted the risks of not having a cohesive ethical governance framework, with projections indicating that a substantial portion of organizations may fail to realize the anticipated value of their AI use cases by 2027.
GenAI amplifies both the positive and negative aspects of data initiatives. It’s projected that as early as 2025, poor data quality, inadequate risk controls, escalating costs, and unclear business value may lead to the abandonment of GenAI projects post-proof of concept.
In her talk, Shinji outlined strategies for enhancing data governance to accommodate the demands of modern GenAI environments.
One key recommendation is to leverage executive dashboards, where core teams track key performance indicators (KPIs).
By analyzing data lineage within these dashboards, organizations can discern the origin of datasets, differentiate between shared and locally specific metrics, and facilitate decentralized management.
Adaptive Data Governance Process
To effectively navigate the integration of GenAI into day-to-day operations, organizations should acquire an adaptive data governance process:
- Define key business outcomes for the organization.
- Identify the data and metrics supporting each outcome.
- Develop the joint ownership and structure of each data and metric.
- Implement governance policy for each ownership segment.
- Establish stewardship through sustainable information architecture.
Companies Are Recognizing Data’s Business Value
Despite the increasing recognition of data's importance, many organizations still struggle to tie data initiatives back to tangible business value.
A notable trend highlighted at the summit was the integration of data teams with finance departments. Data has transitioned from being viewed solely as a technological asset to becoming a component of profit and loss discussions.
Shinji said the summit included talks about strategy implementation, tips and tricks, and best practices to help data teams drive change from a business value perspective.
For Mona, the summit provided valuable insights into engaging with the business, understanding value drivers, and leveraging data to drive strategic decision-making.
.png)
Data Observability Is More Important Than Ever
Mona said the summit put considerable emphasis on the importance of data observability, particularly in the context of GenAI adoption.
The primary challenge faced by organizations venturing into GenAI initiatives is data quality. A staggering 36% of companies say their teams spend days grappling with data issues. As data pipelines become more complex and data increases in volume and velocity, maintaining data quality becomes even more daunting.
Surprisingly, only 7% of teams are equipped to detect data problems before they manifest as business impacts, indicating a critical gap in data observability practices.
Mona said the approach to resolving quality issues depends on whether the data team knows to look for them.
- Known-Unknowns: Typically involve operational errors within data pipelines that can be identified and rectified through data observability practices.
While the tech team plays a key role in validating and addressing these issues, collaboration with the business team is necessary to ensure the accuracy and relevance of data.
- Unknown-Unknowns: Characterized by out-of-range values and drift over time. Traditional rule-based detection methods are insufficient for identifying unknown-unknowns, necessitating the integration of machine learning algorithms into data observability frameworks.
Automating and orchestrating data observability processes are essential for proactively identifying and mitigating these hidden data quality issues.
Technology Can Augment the Data Quality Process
This six-step approach to streamlining data quality processes was outlined at the summit:
- Connect: Autodiscovery of metadata to catalog assets and capture lineage
- Profile: Autoprofile data to collect statistics through semantic detection and domain discovery
- Classify: Autodiscover and classify sensitive data (i.e., PII data).
- Apply rules: Autorules assignment for commonly used rules, with suggestions based on user actions
- Transform/Enrich: Autoclean, transform or enrich data based on rules and assist with suggestions based on user actions
- Monitor: Monitor the business rule usage and exceptions; autonotify of quality issues and potential impact
Shinji added a bit of context she believed was missing from those steps: metadata change detection and management.
Data issues often arise due to unauthorized changes in metadata, such as column deletions or modifications. Automated detection of metadata changes and proactive communication ensure data consistency and integrity across the organization.
“At least having this monitoring part of the schema changes that happened within the platform to notify the owners and the impacted downstream users and owners is so important,” Shinji said.
Modernize Your Data Governance Processes
As data volumes grow exponentially and technologies like GenAI reshape the way we operate, traditional approaches to data governance prove inadequate. To see a modern approach in action, start a free trial of Select Star or request a demo today.
