BI Data Fabric.pdf

ELC475 - Business Intelligence Data Fabric  What is Data Fabric? Data fabric is an end-to-end data integration and management solution, consisting of architecture, data management and integration software, and shared data that helps organizations manage their...

ELC475 - Business Intelligence Data Fabric  What is Data Fabric? Data fabric is an end-to-end data integration and management solution, consisting of architecture, data management and integration software, and shared data that helps organizations manage their data. A data fabric provides a unified, consistent user experience and access to data for any member of an organization globally and in real-time. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 2  What is Data Fabric? Data fabric is designed to help organizations solve complex data problems and use cases by managing their data—regardless of the various kinds of applications, platforms, and locations where the data is stored. Data fabric enables frictionless access and data sharing in a distributed data environment. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 3  Why Use a Data Fabric? Any data-centric organization needs a holistic approach that overcomes the hurdles of time, space, different software types, and data locations. Data needs to be accessible to users who need it, not locked away behind firewalls or located bit by bit in a range of locations. Businesses need to have a secure, efficient, unified environment, and future-proof data solution in order to thrive. A data fabric provides this. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 4 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 5  Why Use a Data Fabric? Data fabric can be visualized as a cloth, spread across the world, wherever the organization’s users are. The user can be at any place in this fabric and still access data at any other location without any constraints, in real-time. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 6  Why Use a Data Fabric? Traditional data integration is no longer meeting new business demands of real-time connectivity, self-service, automation, and universal transformations. Even though collecting data from various sources is not usually the problem, many organizations cannot integrate, process, curate, and transform data with other sources. This crucial part of the data management process needs to happen to deliver a comprehensive view of customers, partners, and products. This gives organizations a competitive edge, allowing them to better meet customer demands, modernize their systems, and harness the power of cloud computing. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 7  What is Data Integration? Data integration is the process of bringing data from disparate sources together to provide users with a unified view. The concept of data integration is to make data more freely available and easier to consume and process by systems and users. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 8  Why Data Integration? Data integration done right can reduce IT costs, free-up resources, improve data quality, and foster innovation all without changing existing applications or data structures. And though IT organizations have always had to integrate, the payoff for doing so has potentially never been as great as it is right now. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 9  Benefits Data Integration Companies with mature data integration capabilities have significant advantages over their competition, which includes: 1. Increased operational efficiency by reducing the need to manually transform and combine data sets 2. Better data quality through automated data transformations that apply business rules to data 3. More valuable insight development through a holistic view of data that can be more easily analyzed 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 10  Data Integration A digital business is built around data and the algorithms that process it, and it extracts maximum value from its information assets—from everywhere across the business ecosystem, at any time it is needed. Within a digital business, data and related services flow unimpeded, yet securely, across the IT landscape. Data integration enables a full view of all the information flowing through an organization and gets your data ready for analysis. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 11  Data Quality Data quality describes the degree to which data fits the purpose it was intended for. Data is considered high quality when it accurately and consistently represents real-world scenarios. To understand this, you have to look at data as being the foundation stone of a hierarchy that is built on it. Over the foundation of data, comes information, which is data placed in context. From actionable information comes knowledge, which develops into wisdom when it is applied. Bad quality data will result in bad information quality, and this moves up the hierarchy, resulting in bad business decisions. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 12  Data Virtualization Data virtualization software acts as a bridge across multiple, diverse data sources, bringing critical decision-making data together in one virtual place to fuel analytics. Data virtualization provides a modern data layer that enables users to access, combine, transform, and deliver datasets with breakthrough speed and cost-effectiveness. Data virtualization technology gives users fast access to data housed throughout the enterprise—including in traditional databases, big data sources, and cloud and IoT systems—at a fraction of physical warehousing and ETL time and cost. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 13 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 14  Data fabrics are used by business application data users users engineers data data data engineers architects stewards 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 15  Data Architects vs. Engineers Data architects conceptualize and visualize data frameworks Data engineers build and maintain them. Data architects guide the Data Science teams while data engineers provide the supporting framework for enterprise data activities. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 16  Data Stewards The role of a Data Steward is specifically tasked with maintaining data control in data governance and master data management initiatives on a day-to- day basis. Data Stewardship is required for data management to succeed. An example of what they may do to achieve this is drafting the data quality rules which their data is measured against. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 17  What is Data Management? Data management enables consistent accessibility, delivery, governance, and security of data to meet an organization’s requirements using tools including master data management, data virtualization, data catalog, and self-service data preparation. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 18 What is Data Management? 19  What is Data Management? With an effective data management solution, organizations can unify all of their data intelligently for better access, trust, and control. This is critical to a business’s success because every effort around improving customer experience, optimizing operations, or transforming an organization relies on harnessing data. To do that successfully, organizations must have a clear understanding of all their data, including metadata, reference data, transactional data, master data, streaming data, and more. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 20  Metadata Management Metadata management is the business discipline of managing the metadata about data. It gives meaning to and describes the information assets in your organization. Metadata unlocks the value of your data by improving that data’s usability and findability. Metadata provides the context required to understand and govern your systems, your data, and your business. By using metadata management, it is easier to find and use data and provide the critical data context your business and IT teams require. 6/7/22 Business Intelligence- Prepared by Nourhan Hamdi 21  What is a Data Catalog? A data catalog is an inventory of a company’s data assets so users can find the information they need fast. The catalog is mostly metadata that provides basic information about other data and describes what it is. Combined with data management and search tools, you have a data catalog. 6/7/22 Business Intelligence- Prepared by Nourhan Hamdi 22  Data Catalog 6/7/22 Business Intelligence- Prepared by Nourhan Hamdi 23  What is Data Governance? Data governance refers to the collection of practices, policies, and roles related to the effective acquisition, management, and utilization of data—ensuring that the data provides as much value as it can within an organization. Data governance confirms the quality and security of a business’s data across the entire organization, determining who can use what data and when. 24  What is Data Governance? Often, data management and data governance are used interchangeably, but this is incorrect. Data management refers more to the technical management of data, whereas data governance refers to the policies of managing data within an organization such as who can use what data and when. 25  Data Security Data security protects company data from internal and external threats and is a critical element of business operations. This encompasses protecting digital information from multiple threats of unauthorized access, data corruption, or theft through its lifecycle. It’s an all-encompassing term that covers hardware and its physical security and includes protection of storage devices, administrative controls, accessibility, and the safety of applications. It also covers company policies and protocols. 26  Data Fabrics To address data silos, complexity, and constant change, data-driven organizations are implementing data fabrics. One of the hottest trends in data management today. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 27  What is a Data Silo? A data silo is a collection of information isolated from an organization and inaccessible to all parts of a company hierarchy. Data silos create expensive and time consuming problems for businesses, but they are relatively simple to resolve. By getting rid of data silos, you can access the right information at the appropriate time, helping you make smart decisions for the business. Removing data silos also reduces information storage costs and duplicate information. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 28  Facts about Data Fabrics 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 29 A data fabric is a modern, distributed data architecture that includes shared data assets and optimized pipelines to address data challenges in a unified way. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 30 Concerns addressed by data fabric: Highly-distributed, difficult-to-access, poorly-integrated business data Business users’ demand for self-served, trusted data New business opportunities needing real-time data Growing data compliance and privacy regulations A need to modernize a data architecture 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 31  Some Benefits of Data Fabric Fuel your data-driven business Accelerate value realization Empower your people Optimize your processes Benefit from technology innovation sooner Save time and money Govern and comply with confidence 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 32 Data Fabric is not a single product or specific platform that you simply buy and deploy. Instead, you build it using a common distributed architecture, metadata management, data integration, and data delivery capabilities that use optimized pipelines. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 33  Data Fabrics aren’t just architecture and single products, and they aren’t rip and replace implementations. A data fabric is not the right solution for every organization. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 34 DFs are currently in the early-adopter phase Current users are selectively implementing key components such as data virtualization and data catalogs to address persistent problems and set the foundation for a more comprehensive implementation. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 35 To build a Data Fabric, organizations should start with a Data Fabric vision and strategy Organizations should prioritize their highest business impact opportunities, perform a data management and integration gap analysis, and take advantage of data virtualization. They should also standardize their approach to shared data assets and organize for agility and productivity. Then prove the business value and scale up. 6/7/22 New Trends in BIS - Prepared by Nourhan Hamdi 36