What DevOps Need to Know About Data Governance

Data governance is an umbrella term encompassing several different disciplines and practices, and priorities often depend on who is leading the effort. Data officers, privacy officers, security officers, and risk managers typically focus on privacy, security, and the regulations that govern data governance programs. Data Scientists, Marketers, DevOps Leaders, and Business Analysts are more likely to focus on proactive data governanceincluding data catalogs, data integration, data qualitydata lineage, customer data profiles and master data management.

There’s a lot to unpack across all terms, practices, and technologies, and some overlapping capabilities and goals. Shams Chauthani, CTO and senior vice president of engineering at Zilliant, agrees that because there are multiple goals, collaboration between business stakeholders, IT teams and data teams is key. the success of the programs.

“Data governance is often treated in a siled way, primarily as an IT-managed compliance requirement,” he says. “In today’s digital age, data is the greatest asset, and treating data governance as a siled effort managed exclusively by IT is a disservice to the entire organization. For organizations to fully deliver on the promise of smarter decision-making with data, the data governance process must be continuously improved with commitments from all stakeholders.

For this article, I consulted with industry experts to identify what leaders and devops teams need to know about data governance and how they can contribute to its goals.

Data governance is a major organizational change

Grant Fritchey, devops advocate at Redgate Software, suggests that stakeholder engagement is just the start. “Data governance is everyone’s responsibility,” he says. “You can’t just assign one person or department the task of data governance and expect to be successful. Data governance needs to become part of what everyone in IT does as part of their job. »

One way to achieve the required collaboration and define responsibilities is to connect data governance directly to employee workflows. John Milburn, CEO of Clear Skye, says, “Data governance is not so much a technical function as it is a people and process function. This is why it is essential that your governance solution aligns with the existing employee workflow, otherwise it is doomed.

Devops teams should look for opportunities to improve data operations, including automations to support data integration, cataloging, and quality. John Wills, Field CTO at Alation, said, “The next generation of data governance is active, catalog-driven governance, which means it’s vital for all knowledge workers, including devops and dataops, and is part of their daily work.

Privacy and data protection rules

Immuta CEO Matthew Carroll shares one of the main reasons for top-down adherence to data governance programs. “We are seeing a major shift in the landscape of data management and governance as more consumers become aware of their privacy rights, thanks in large part to new regulations. As a result, organizations are wondering how best to protect data assets and comply with privacy regulations while trying to scale and derive value from their data faster, which ultimately requires policies. digitized data and automated data access and security in the cloud.

Barr Moses, CEO and co-founder of Monte Carlo, agrees. “Data governance is more critical than ever as businesses ingest more data and greater data regulations like GDPR and CCPA roll out. We can tackle these compliance headwinds before they slow us down by making data more accessible, meaningful, compliant and trusted. »

Leaders must trust data to make decisions

Beyond compliance considerations, the next level of importance that drives data governance efforts is trust that data is accurate, timely, and compliant with other data quality requirements.

Moses has several recommendations for technical teams. She says, “Teams need to have visibility into critical charts and reports and treat data integrity like a first-class citizen. True data governance must go beyond defining and mapping data to truly understanding how it is used. An approach that prioritizes observability in data can provide collective meaning around specific analytics use cases and enable teams to prioritize the data most important to the business.

Kirk Haslbeck, vice president of data quality at Collibra, shares several best practices that improve overall trust in data. He says, “Trusted data starts with data observability, using metadata for context, and proactively monitoring data quality issues. While data quality and observability establishes that your data is usable, data governance ensures that its use is streamlined, secure, and compliant. Data governance and data quality must work together to create value from data. »

Data as competitive differentiation

Once there is basic trust in data, business leaders want to use data, analytics, and machine learning to transform the business. Haslbeck continues, “Every business seeks data for competitive advantage, and data governance and quality should be a priority.”

How does data governance create competitive advantage? John Wheeler, Senior Risk and Technology Advisor at AuditBoard, explains: “Data governance is now a strategic priority for organizations seeking new digital products and services for growth. As such, data governance requires strong leadership from chief data officers or chief digital officers who understand the need for data consistency, quality, transparency, and accuracy.

One area of ​​differentiation occurs in open data models for access management in business-to-business partnership models. For instance, open bank and the Financial Grade API aims to secure the exchange of data and banking systems to an ecosystem of developers, fintech providers and partners.

Brook Lovatt, Product Manager at Cloudentity, shared the specifics of open data models with me. He says, “There are more opportunities to evolve and innovate in open data governance models than in closed models, but open models also require a new set of security and compliance considerations. These open data specifications provide models and protocols that determine how systems communicate with each other, allowing data to flow between applications, services, platforms and providers.

Develops practices that support data governance

Data privacy, security, quality, and reliability are all reasons why data governance is important for data-driven organizations. Here are some recommendations for leaders and devops teams.

  • Wills says DevOps teams should take an active role in setting up and updating data catalogs. “The Data Catalog is an enterprise reference system containing a web of related knowledge, including technical assets such as tables, columns, queries, and models, and non-technical assets such as glossaries and The result is a rich inventory of high-quality assets and contextual knowledge that is trusted and drives productivity through powerful search, reuse, collaboration, and crowdsourcing.
  • Steve Jones, devops attorney at Redgate Software, recommends instituting data governance as part of “regular work to ensure that when changes are made to schemas, new data is collected and data is classified. and protected appropriately”.
  • Milburn recommends “integrating your governance solution with your IT service management platform as a smart way for businesses to protect their data while providing employees with a familiar user experience.”
  • Eldad Chai, CEO and co-founder of Satori Cyber, says data security operations should be part of the data governance plan. “As big data evolves and multi-cloud environments grow in popularity, high-growth businesses need a non-disruptive, compliant way to manage their technology stacks and the sensitive data stored therein. A modern data governance strategy such as data security operations (datasecops) enables businesses to regain control, reduce risk, maintain compliance, and provide secure access to data to make better business decisions. automatically and easily.

Data governance, along with continuous testing and left-shifted security practices, are key disciplines that DevOps teams should build into the structure of their application architectures and development processes. Viewing these practices as an afterthought can lead to business risk, technical debt, and missed innovation opportunities.

Copyright © 2022 IDG Communications, Inc.