Global Data Quality Standard PDF
Document Details
Uploaded by Deleted User
Tags
Summary
This document details the Global Data Quality Standard (DC-DG-01-03), outlining the requirements, roles, and responsibilities for effective data quality management at Walmart. It covers definitions of key terms, roles, responsibilities, and a data quality lifecycle.
Full Transcript
**Global Data Quality** **Standard** **DC-DG-01-03** **Section 1: Purpose/Overview** The Global Data Quality Standard specifies the requirements, roles, and responsibilities for effective data quality management, ensuring trusted data that is 'fit for use'. **Section 2: Target Audience** This s...
**Global Data Quality** **Standard** **DC-DG-01-03** **Section 1: Purpose/Overview** The Global Data Quality Standard specifies the requirements, roles, and responsibilities for effective data quality management, ensuring trusted data that is 'fit for use'. **Section 2: Target Audience** This standard applies to associates of Walmart Inc., its subsidiaries, and any operating units in which Walmart has a majority or controlling interest ("Walmart"). **Section 3: Detailed Requirements** **3.1** **Definitions** A. **Business Data Owner** is accountable for the business vision and strategy for data use within its Business Unit. The BDO function is typically fulfilled by an officer withing a Business Unit. Global Data Governance Policy, DC-DG-01 B. **Business Data Steward** is responsible for executing the business vision and strategy for data use within its Business Unit. The BDS function is typically fulfilled by Product Owners, Data Analysts, or associates designated as Data Stewards. Global Data Governance Policy, DC-DG-01 C. **Technical Data Steward** is responsible for working to ensure that the technology used by Walmart is in alignment with business vision as well as the policies, standards, and processes of Digital Citizenship and Global Tech. The TDS function is typically filled by Global Tech associates who are Technology Data Architects or Data Modelers, Data Engineering Leads, and designated Data Custodians. Identity, Authentication, and Access Controls Policy, GTPG-18-P D. **Business Unit** is a group of Walmart associates and any third parties responsible for performing a business function, such as Finance, H.R, or Walmart+. Global Data Governance Policy, DC-DG-01 E. **Production Data** refers to the actual data generated, utilized, or stored within the production or live environment of a solution or a system during daily business operations. It encompasses various forms of data collected, processed, and stored as part of operational activities, such as customer or product information, transaction records, logs, sales data, etc. This data drives businesses\' decision-making, operational optimization, and trend identification. Given its sensitive and operational nature, production data is subject to stringent InfoSec, Privacy, and Data Governance Compliance controls. Global Data Access Standard, DC-DG-01-05 F. **Non-Production** Data refers to data generated, utilized, or stored outside of the production or live environment, typically within the development, testing, staging, and backup environments. Global Data Access Standard, DC-DG-01-05 G. **Data Quality** refers to the accuracy, consistency, and reliability of data collected and used by Walmart to ensure effective decision-making, accurate analysis, and efficient operations. **3.2** **Roles and Responsibilities** A. **Business Data Owner must:** 1. Accountable for data quality management in accordance with Walmart's global data governance policies, standards, and processes. 2. Accountable for ensuring domain data quality is measured, tracked, and improved in collaboration with the Business Data Steward. 3. Prioritize metric integrity and business continuity in the business strategy. 4\. Approve proposed data quality targets, based on data criticality.\ 5. Ensure that issues for remediation are quickly resolved. B. **Business Data Steward must:** 1. Define data quality rules, which include a business statement and technical data quality rule definition. 2. Drive data quality requirements, definitions, assessments, and remediations in collaboration with Technical Data Stewards and other SME. 3. Determine testing requirements in collaboration with the Technical Data Steward that support metric integrity and business continuity. 4. Propose data criticality assessments and associated data quality targets. 5. Determine if there were potential issues discovered that require remediation. 6. Collaborate with Technical Data Steward to design a remediation plan. 7. Ensure that issues discovered are successfully remediated. a. Responsible for data remediation activities at the source level. 8. Escalate issues as need to the Business Data Owner C. **Technical Data Steward must:** 1. Responsible for the development and implementation of data quality rules as defined by the Business Data Steward 2. Responsible for implementing test cases and other quality checks in collaboration with the Business Data Stewards to identify or mitigate errors throughout the data transformation pipeline. 3. Responsible for creating and automating a framework for testing metric integrity. 4. Build and deploy data quality dashboards. 5. Support Business Data Stewards to remediate uncovered issues. **3.3 Measuring Data Quality** A. Data quality must be measured across various dimensions such as completeness, conformity, integrity, and timeliness as shown in the following chart: +-----------------------+-----------------------+-----------------------+ | **DQ Dimension** | **Description** | **Validation** | +=======================+=======================+=======================+ | **Conformity** | Ensures data conforms | 1\. Does data | | | to internal standards | conform to defined | | | and external | formats, standards, | | | regulations while | and business rules? | | | maintaining | | | | correctness and | 2\. Are the data | | | precision. | values stored for | | | | an object, the | | | | correct values? | | | | | | | | [Examples]{.underline | | | | } | | | | | | | | 1\. Ensure date | | | | format -- | | | | mm/dd/yyyy | | | | | | | | 2\. Ensure quantity | | | | available is not | | | | negative | | | | | | | | 3\. Does date field | | | | contain date or is | | | | alphanumeric? | +-----------------------+-----------------------+-----------------------+ | **Integrity** | Ensures data | 1\. What data is | | | uniqueness, accuracy, | missing that is | | | and consistency | needed to identify | | | across all systems. | relationship | | | | linkages? | | | | | | | | 2\. Is data uniform | | | | across different | | | | systems and | | | | datasets, without | | | | conflicting | | | | information? | | | | | | | | 3\. Do multiple | | | | copies of the data | | | | exist that | | | | represent the same | | | | underlying data? | | | | | | | | [Example] | | | | | | | | 1\. Are there | | | | duplicate records | | | | of the same data? | | | | | | | | 2. Product-Inventory | | | | relationship, i.e. | | | | POS system: Customer | | | | purchases item, POS | | | | system should | | | | immediately update | | | | the inventory count | | | | accordingly. | +-----------------------+-----------------------+-----------------------+ | **Timeliness** | Focuses on keeping | 1\. Is the data up | | | data current and | to date and | | | timely. | available at the | | | | time needed? | | | | | | | | [Example] | | | | | | | | 1\. Transactions | | | | that are older than | | | | 60 days do not need | | | | to appear in the | | | | operational | | | | dashboard because | | | | they are purged | | | | from the source. | +-----------------------+-----------------------+-----------------------+ | **Completeness** | Ensures that all | 1\. What data is | | | required data fields | missing or does the | | | are filled with data. | data contain gaps? | | | | | | | | [Example] | | | | | | | | 1\. All records must | | | | have a value | | | | populated in the | | | | CustomerName field. | +-----------------------+-----------------------+-----------------------+ **Section 4: Compliance** A. Any violation of this policy document may result in disciplinary action up to and including termination and may be referred to the appropriate law enforcement authorities when applicable. B. Walmart's non-enforcement of any policy or standard does not constitute a waiver of its terms. Walmart may, at its discretion, choose to enforce the provisions at any time and without prior notice. **Section 4: Contact Information** For additional information about this document or to request a content change, contact the Data Policies and Standards team via [ServiceNow](https://walmartglobal.service-now.com/wm_sp?id=wm_sc_cat_item&sys_id=7888d350db385784d811568bdc9619fe) or at. **Section 6: Resources** - Global Data Governance Policy, [DC-DG-01](https://one.walmart.com/content/uswire/en_us/work1/policies/non-people-policies/data-governance/global-data-governance-dg-01.html) - Global Data Classification Policy, [[DC-DG-03]](https://one.walmart.com/content/uswire/en_us/work1/global-governance/digital-citizenship/data-policies/data_governance/dg-standards/global-data-usage-classification-standard.html) - [Global Records and Information Management Policy, DC-DG-09](https://one.walmart.com/content/uswire/en_us/work1/policies/non-people-policies/privacy-and-records/walmart-global-records-and-information-management-policy--dc-dg-/records-management.html) - [Global Data Roles and Responsibilities Standard, DC-DG-01-01](https://one.walmart.com/content/uswire/en_us/work1/global-governance/digital-citizenship/data-policies/data_governance/dg-standards/dg-02-st-01.html) - Global Data Access Standard, [DC-DG-01-05](https://one.walmart.com/content/uswire/en_us/work1/global-governance/digital-citizenship/data-policies/data_governance/dg-standards/dg-01-st-01.html) - Global Critical Data Element Guideline, [DC-DG-01-00-02](https://one.walmart.com/content/uswire/en_us/work1/global-governance/digital-citizenship/data-policies/data_governance/dg-guidelines/cde-guideline.html) - [Global Data Lineage Guideline, DC-DG-01-04-01](https://one.walmart.com/content/uswire/en_us/work1/global-governance/digital-citizenship/data-policies/data_governance/dg-guidelines/data_lineage.html) - [Change Management Policy Group, GTPG-07](https://one.walmart.com/content/uswire/en_us/work1/technology/global-tech/tech-policy-digital-library/07-policy-group.html) - [Disaster Recovery Policy Group, GTPG-10](https://one.walmart.com/content/uswire/en_us/work1/technology/global-tech/tech-policy-digital-library/10-policy-group.html) - Identity, Authentication, and Access Controls Policy, [GTPG-18-P](https://one.walmart.com/content/uswire/en_us/work1/technology/global-tech/tech-policy-digital-library/18-policy-group/18_policy.html) - [DataPedia](https://one.walmart.com/content/uswire/en_us/work1/technology/datapedia-overview.html) **Appendix -- A Data Quality Lifecycle** Data quality must be managed across its lifecycle, from creation or ingestion to disposal. This includes managing data as it moves within and between systems. To achieve high data quality, a systematic approach that spans across various stages, including data creation, collection, processing, storage, management, and data retention and disposal must be adopted. Data must be properly validated, cleaned, and formatted before it is ingested into systems and there must be processes in place to detect and correct errors or inaccuracies. Picture of the Data Quality Lifecycle. Beginning with Creation, then Collection, Processing, Storage, Disposal, Interpretation, Analysis and lastly, Management.