Data Warehousing Considerations PDF

Data Warehousing Considerations in Building your Datawarehouse Data Warehouse INTRODUCTION Building a data warehouse is a complex yet essential process for organizations looking to consolidate data and make strategic business decisions. This topic explores the critical phases and considerat...

Data Warehousing Considerations in Building your Datawarehouse Data Warehouse INTRODUCTION Building a data warehouse is a complex yet essential process for organizations looking to consolidate data and make strategic business decisions. This topic explores the critical phases and considerations in developing a robust data warehousing system. From planning and design to deployment and maintenance, the focus is on aligning technical solutions with business needs, ensuring data quality, and leveraging best practices for success. This comprehensive guide addresses key factors such as infrastructure selection, data transformation, and ongoing performance management to create a scalable and efficient data warehouse tailored to organizational goals.  PROJECT LIFE CYCLE STEPS AND CHECKLISTS Project Planning Requirements Definition Design Construction Deployment Maintenance CRITICAL FACTORS FOR SUCCESS Do not launch the data warehouse unless and until your company is ready for it. Find the best executive sponsor. Ensure continued, long-term, and committed support. Emphasize the business aspects, not the technological ones, of the project. Choose a business-oriented project manager. Take an enterprise-wide view for the requirements. Have a pragmatic, staged implementation plan. Communicate realistic expectations to the users; deliver on the promises. Do not overreach to cost-justify and predict ROI. Institute appropriate and effective communication methods. Throughout the project life cycle, keep the project as a joint effort between IT and users. Adopt proven technologies; avoid bleeding-edge technologies. Recognize the paramount importance of data quality. Do not ignore the potential of data from external sources. Do not underestimate the time and effort for the data extraction, transformation, and loading (ETL) functions. Select the architecture that is just right for your environment; data warehousing is not a one-size-fits-all proposition. Architecture first, technology next, and only then, tools. Determine a clear training strategy.  GUIDELINES FOR EVALUATING VENDOR SOLUTIONS Here are a few practical guidelines. First and foremost, determine the functions in your data warehouse that absolutely need vendor tools and solutions. For each type of product you need, carefully list the features that are expected. Divide the features into groups by importance—high, medium, and low. Use these groups of features to grade the products you are considering. Allocate enough time to research available solutions and vendors thoroughly. If you try to incorporate solutions from too many different vendors, you must be prepared to face serious challenges of incompatibilities and restrictions for integration. Stay with two or three vendors whose products are most appropriate for your needs. Metadata is a key component of a data warehouse. Ensure that the vendor products you choose can handle metadata satisfactorily. The standing and stability of the vendor are equally important as the effectiveness of the products themselves. Even when the products are suitable for your environment, if you are concerned about the staying power of the vendor, have second thoughts on selecting these products. Never rely solely on vendor demonstrations as the basis for your selection, nor should you check only the references furnished by the vendors themselves. Test the tools and products in your environment, with subsets of your own data. Arrange for both user representatives and IT members of the project team to test the products, jointly and independently. Establish a definitive method for comparing and scoring competing products. You may devise a point system to score the various features you are looking for in a product type. The success of your data warehouse rides on the end-user tools. Pay special attention to the choice of the end-user tools. You may compromise a bit on the other types, but not on the end-user tools. Most of your end-users will be new to data warehousing. Good, intuitive and easy to use tools go a long way in winning them over. Users like tools that seamlessly incorporate online queries, batch reporting, and data extraction for analysis.  HIGHLIGHTS OF VENDORS AND PRODUCTS Here is a list of vendors and products for data warehousing/business intelligence Actuate (www.actuate.com) -BIRT (Open Source Business Intelligence and Reporting Tools) ADVIZOR Solutions, Inc. (www.advizorsolutions.com) -ADVIZOR Analyst agileDSS Inc.(www.agiledss.com) –agileWORKFLOW Applix, Inc. (www.applix.com) -TM1 DataFlux(www.dataflux.com) -DataFluxData Quality Integration Platform Hummingbird Ltd. (www.hummingbird.com) -BI/Query IBM (www.ibm.com) -For this large vendor, refer to the vendor’s official Website for products and solutions. iDashboards(www.iDashboards.com) -iDashboardsEnterprise Edition Teradata (www.teradata.com) -For this large vendor, refer to the vendor’s official Website for products and solutions. Visual Mining (www.visualmining.com) –NetChartsPerformance Dashboards (NCPD) REAL-WORLD EXAMPLESOF BEST PRACTICES AIRLINES SPECIALTY TEXTILES TRAVEL HEALTH CARE SECURITIES INTERNATIONAL SHIPPING AND DELIVERY RAIL SERVICES PHONE SERVICE HOME IMPROVEMENT RETAIL CREDIT UNION LIFE INSURANCE TELECOMMUNICATIONS References: DATA WAREHOUSING FUNDAMENTALS FOR IT PROFESSIONALS (2ndEdition) - PAULRAJ Adamson,PONNIAH Christopher, and Michael Venerable, DataWarehouseDesign Solutions, New York: McGraw-Hill, 1996. Wiley,1998. Humphries, Mark, et al., Data Warehousing: Architecture and Implementation, Upper Saddle River, NJ: Adriaans, Pieter, and Doff Zantinge, Data Mining, Harlow, UK: Addison Wesley, 1996. Prentice-Hall PTR, 1999. Ambler, Scott W., Agile Database Techniques, Hoboken, NJ: Wiley, 2003. Inmon, W. H., Building the Data Warehouse (2nd Edition), New York: Wiley, 1996. Barquin, Ramon C., and Herb Edelstein, Planning and Designing the Data Warehouse, Upper Inmon, W. H., et al., Data Warehouse Performance, New York: Wiley, 1999. Saddle River, NJ: Prentice-Hall PTR, 1997. Inmon,W. H., et al., Managing the Data Warehouse: Practical Techniques for Monitoring Operations and Berson, Alex, and Stephen J. Smith, Data Warehousing, Data Mining, and OLIP, New York: McGraw- Performances, Administering Data and Tools, Managing Change and Growth, New York: Wiley, 1997. Hill, 1997. Kelly, Sean, Data Warehousing in Action, Chichester, UK: Wiley, 1997. Berry, Michael J. A., and Gordon Linoff, Data Mining Techniques, New York: Wiley, 1997. Kimball, Ralph, and Richard Merz, The Data WebhouseToolkit: Building the Web-Enabled Data Warehouse, Bigus, Joseph P., Data Mining with Neural Networks, New York: McGraw-Hill, 1996. New York: Wiley, 2000. Bischoff, Joyce, and Ted Alexander, Data Warehouse: Practical Advice from the Experts, Upper Kimball, Ralph, The Data Warehouse Toolkit: Practical Techniques for Building Dimensional Data Warehouses, Saddle River, NJ: Prentice-Hall, 1997. New York: Wiley, 1996. Cabena, Peter, et al., Discovering Data Mining: From Concept to Implementation, Upper Saddle Kimball, Ralph, et al., The Data Warehouse Lifecycle Toolkit: Expert Methods for Designing, Developing, and River, NJ: Prentice-Hall PTR, 1998. Deploying Data Warehouses, New York: Wiley, 1998. Codd, E. F., Providing On-Line Analytical Processing to User Analysts, San Jose: Codd & Date, Inc., Malone, Thomas W., and Rockart, John F., Computers, Networks, and the Corporation, Center for Information 1993. Systems Research, Sloan School of Management, Massachusetts Institute of Technology, August 1991. Craig, Robert S., et al., Microsoft Data Warehousing, New York: Wiley, 1999. Mattison, Rob, Data Warehousing Strategies, Technologies, and Techniques, New York: McGraw-Hill, 1996. Debevoise, Nielson Thomas, The Data Warehouse Method: Integrated Data Warehouse Support Meyer, Don, and Casey Cannon, Building a Better Data Warehouse, Upper Saddle River, NJ: Prentice-Hall Environments, Upper Saddle River, NJ: Prentice-Hall PTR, 1999. PTR, 1998. Devlin, Barry, Data Warehouse from Architecture to Implementation, Boston, MA: Addison Wesley, Morse, Stephen, and David Isaac, Parallel Systems in the Data Warehouse, Upper Saddle River, NJ: Prentice- 1997. Hall PTR, 1998. Eckerson, Wayne, Performance Dashboards, Hoboken, NJ: Wiley, 2006. Poe, Vidette, et al., Building a Data Warehouse for Decision Support (2nd Edition), Upper Saddle River, NJ: Eckerson, Wayne, Dashboard or Scorecard: Which Should You Use? Date Warehousing Institute, Prentice-Hall PTR, 1998. January 2005, www.tdwi.com. Singh, Harry, Interactive Data Warehousing, Upper Saddle River, NJ: Prentice-Hall PTR, 1999. English, Larry P., Improving Data Warehouse and Business Information Quality: Methods for Thomsen, Erik, OLAP Solutions: Building Multidimensional Information Systems, New York: Wiley ,1997. Reducing Costs and Increasing Profits, New York: Wiley, 1999. Westphal, Christopher, and Teresa Blaxton, Data Mining Solutions, New York: Wiley, 1998. Gill, Harjinder S., and Prakash C. Rao, The Official Client/Server Computing Guide to Witten, Ian H., and EibeFrank, Data Mining, San Francisco, CA: Morgan Kaufmann, 2005. DataWarehousing, Indianapolis, IN: Que Corporation, 1996. Groth, Robert, Data Mining: A Hands-On Approach for Business Professionals, Upper Saddle River, NJ: Prentice-Hall PTR, 1998. Hammergren, Tom, Data Warehousing Strategies, Technologies, and Techniques, New York:  Q and A Concerns in Final Requirements