HUM 4072 - Unit 2_ Data Analytic Thinking PDF

HUM 4072: Fundamentals of Business Analytics UNIT - 2 DATA-ANALYTIC THINKING FOR BUSINESS Prepared By: Dr. Vineetha E Jathanna Assistant Professor Department of Humanities & Management What is A Business Enterprise??? A business...

HUM 4072: Fundamentals of Business Analytics UNIT - 2 DATA-ANALYTIC THINKING FOR BUSINESS Prepared By: Dr. Vineetha E Jathanna Assistant Professor Department of Humanities & Management What is A Business Enterprise??? A business enterprise consists of producing goods or services in exchange for commercial and financial benefits. A business enterprise is any type of operation that is involved in providing goods or services with the anticipated outcome of earning a profit. Business enterprises can vary in size, industry, and structure, but they all share common functions that are essential for their successful operation. Types of Business Enterprises Broad Perspective Primary sector The primary sector involves businesses that are at the beginning of the production processes. These businesses make sure that the raw materials are created and produced to be used later by other companies. Secondary sector The secondary sector consists of business enterprises at the second step of the production process. These businesses use raw materials produced from the primary sector to develop into new goods and services. Tertiary sector  Ownership and Management Perspective 1) Private Sector Enterprises: Enterprises owned, controlled and managed by private individuals fall under this category with the main objective of earning. A) Sole proprietorship B) Participation C) Joint Hindu Family Business D) Cooperative E) Company 2) Public Sector Enterprises: Business enterprises owned, controlled and operated by public enterprises, with the primary goal as welfare and secondary goal as profit. Either whole or most of the investment in these ventures is done by the government such as: A) Departmental undertaking B) Public corporation C) Government companies 3) Joint Sector Enterprises: The joint sector is a form of partnership between the private sector and the government where management is generally in the hands of the private sector, and adequate representation by the government on the board of directors. Resources in such enterprises are mostly generated equally. Thus, one of the first decisions an entrepreneur must make for his new venture is how the business should be structured. From the entrepreneur's point of view, the most commonly chosen forms for starting a new venture are: Sole proprietorship Partnerships Company Functions of a Business Enterprise The basic functions of a business enterprise are Finance, Operations, Human Resources, and Marketing.  Finance One of the essential functions of a business is raising and managing money. A business enterprise may use internal or external sources of finance to raise the funds needed to get the business going. Internal sources of finance involve the money that business owners invest in their own business. In contrast, external sources of finance involve cash from outside sources, such as money from family, banks loans, and investors. After the money starts moving around the business, the business managers should manage it cautiously so they don’t have too many costs, thereby failing to make any sales.  Operations An important function of a business enterprise is the use of raw materials to produce new goods that will be served to customers. A business also uses its resources to provide services to customers. A business enterprise is always concerned with producing types of goods or offering services that meet the needs and demands of customers. If this need or demand is not met or is relatively small, there is no real purpose for production.  Human Resources Another important function of a business enterprise is that of human resources. A business needs to get the right human capital to provide goods or services. This entails hiring people with the necessary expertise and skillset that the production process requires.  Marketing Marketing is concerned with commercializing the goods and services a business offers. This includes pricing strategies, strategizing the way customers are approached, and determining why someone would want to buy the good or service.  ERP - ENTERPRISE RESOURCE PLANNING Enterprise Resource Planning software is fully integrated “Business Management Software” to link business processes automatically and give real time information to authorized user. Key Characteristics Seamless integration of all the information flowing through a company. To support business goals. Integrated, secure, self-service processes for business. Lower Costs. Empower Employees.  MOBILE INTEGRATION  BENEFITS OF ERP TANGIBLE BENEFITS INTANGIBLE INVENTORY REDUCTION BENEFITS PERSONAL REDUCTION INFORMATION VISIBILITY PRODUCTIVITY IMPROVEMENTS CUSTOMER RESPONSIVENESS ORDER MANAGEMENT IMPROVEMENTS COST REDUCTIONS FINANCIAL CYCLE IMPROVEMENTS INTEGRATION INFORMATION TECHNOLOGY COST REDUCTION STANDARDIZATION PROCUREMENT COST REDUCTION FLEXIBILITY CASH MANAGEMENT IMPROVEMENT BETTER PERFORMANCE REVENUE / PROFIT INCREASE EXAMPLE: ERP - MODULES MANUFACTURING / PRODUCTION MODULE: Production Planning helps an organization plan production with the optimum utilization of all available resources. Material Requirement Planning is done based on the production advice generated by the sales department. Feasibility of production is evaluated using details like raw material availability and procurement time, machine availability and capacity. The important sections in manufacturing module will be: » Resource & capacity planning » Material planning » Workflow management » Quality control » Bills of material » Manufacturing process Production Order Execution The production order is the main central data object in shop floor control and manufacturing execution process. The production order contains every data relevant to the production objectives, material components, required resources, and costs. A normal production order will cover the demand for a single material or product, but you can also produce multiple products jointly in one production order (co-products) and distribute incurred costs between the different products. The complete process includes several steps: Creation and release of a production order Goods issues of components Confirmation of production activity Goods receipts of the finished good Production Order A production order defines which material is to be processed, at which location, at what time and how much work is required. It also defines which resources are to be used and how the order costs are to be settled. Production Order Statuses Created: Order is created and changes can be made Released: Order released, suggested not to make any change Partially confirmed: Production qty/activity partially confirmed Fully confirmed: Production qty/activity fully confirmed Delivery completed: Entire order quantity has been received Technically completed: Order processing is over, ready for month-end processing by accounts 5 Steps involved in Manufacturing Procedure: 1.View Orders 2.Make Product 3.Record consumption & output 4.Record process data & batch characteristics 5.Final Confirmation & Back Flush  FINANCE / ACCOUNTING MODULE: Finance Module take care of all accounts related entries and their impact on the whole system. How the finance comes and how it is been utilized. Total flow of money (Cash/Bank) and total expenditures will be reflected here. As an after effect of this, the management will be able to take their important financial decision, Budgeting etc. They can come to know about company’s financial position at any point of time. All sorts of important financial reports i.e. Trial Balance, Trading A/c, Profit & Loss A/c, Balance Sheet, Debtor’s Balance, Creditors Balance, Cash/Bank Fund position and many more are covered in this module. The important sections in finance module will be: Accounts payable Accounts receivable General ledger Cash management Inventory Module The inventory module will be provided with facilities to handle receipts, transfers, returns, sales and issues of stock with full stock-take and stock adjustment functionality, providing management control over the quantity and value of stock on hand. Full transaction history in both detail and summary will allow management to spot trends, analyze sales and profitability while preventing over- stocking and ensuring that customer demands are met without lost sales. Human Resource HR module will maintain a complete employee database including contact information, salary details, attendance and payroll of all employees. Also the processing of the pay roll with respect to attendance can be done in this module. Recruitment Benefits The important sections in HR moduleCompensations will be: Training Payroll Time and attendance Labor rules People management Sales Module: ERP Sales module is the most important and essential function for the existence of an organization. ERP Sales module implements functions of order placement, order scheduling, shipping and invoicing. Capturing enquiries, order placement, order scheduling and then dispatching and invoicing form the broad steps of the sales cycle. important analysis reports are provided to guide decision making and strategy planning. Purchase Module: ERP Purchasing module streamline procurement of required raw materials. It automates the processes of identifying potential suppliers, negotiating price, awarding purchase order to the supplier, and billing processes. All purchasing activities such as supplier evaluation, placing purchase order, order scheduling and billing are covered in this module.  HOW ERP REDUCES YOUR COSTS? Cut operations costs: Seeing exactly where your money is going is the first step to cutting costs. ERP provides this information both at a summary level and at a detail level. Complete visibility on a monthly, quarterly, or annual basis can provide graphical, up-to-the- minute information to allow for timely adjustments. Less data entry and errors: ERP organizes all of your company's information into one, centralized system. This means that there is no need for different departments to re-key information and less need for manual paperwork, thus reducing the potential for errors. Reduce purchasing costs: By forecasting demand to suppliers, taking better advantage of quantity breaks, and tracking vendor performance with ERP, you can get the best prices from your vendors. Return on Investment: ERP provides solid, measurable financial benefit within the first year after implementation. Inventory carrying costs decrease due to better planning, tracking and forecasting of requirements. Vendor pricing decreases by taking better advantage of quantity breaks and tracking vendor performance. Collections are turned faster due to better visibility into accounts and fewer billing errors. In Short we can say that ERP provides: More centralized and efficient operations, with resulting cost savings. Easier integration of new applications and functionality. A more comprehensive and current (real-time or near real-time) view of the business.  FEW ERP SOFTWARE SAP ERP: Is one of the largest and most well-established ERP vendors globally. Their ERP suite offers comprehensive solutions for various industries and business sizes. ORACLE ERP CLOUD provides a range of ERP solutions, including cloud-based options. their ERP cloud offers a scalable and flexible platform for businesses of different sizes. MICROSOFT DYNAMICS 365: Microsoft offers a suite of ERP solutions under the dynamics 365 umbrella. these solutions cater to different aspects of business operations, including finance, supply chain, and customer relationship management. TALLY.ERP 9 is widely used in india, especially by small and medium-sized businesses. it's known for its user-friendly interface and suitability for accounting  Infor ERP offers industry-specific ERP solutions that cater to various sectors such as manufacturing, distribution, healthcare, and more. Ramco ERP provides cloud-based ERP solutions that cover a wide range of functionalities, including finance, HR, supply chain, and more. Zoho ERP offers a suite of business software, including ERP modules, which can be suitable for small and medium-sized businesses. Netsuite now a part of Oracle, provides cloud-based ERP solutions that are scalable and offer comprehensive features for various industries. Sage ERP offers ERP solutions that focus on accounting and financial management, along with other business processes. Focus ERP : Focus Softnet provides ERP solutions tailored for various industries, including manufacturing, distribution, and retail.  CRM - CUSTOMER RELATIONSHIP MANAGEMENT Customer relationship management (CRM) is a model for managing a company’s interactions with current and future customers. It involves using technology to organize, automate, and synchronize sales, marketing, customer service, and technical support. CRM “is a business strategy that aims to understand, predict and manage the needs of an organisation’s current and potential customers” DEFINITION “CRM is concerned with the creation, development and enhancement of individualised customer relationships with carefully targeted customers and customer groups resulting in maximizing their total customer life-time value”. THE PURPOSE OF CRM  Help a business to keep customers.  It helps the business to understand what it needs to do to get more customers.  Reduce costs by managing costly complaints and finding out what services are useless for customers.  Help a company figure out if its product is working and, ultimately, increases profit.  Prime reason is to log and manage customer relationships.  RELATIONSHIP MARKETING Relationship marketing was first defined as a form of marketing developed from direct response marketing campaign which emphasizes customer retention and satisfaction, rather than a dominant focus on sales transactions. Marketing activities that are aimed at developing and managing trusting and long-term relationships with larger customers. In relationship marketing, customer profile, buying patterns, and history of contacts are maintained in a sales database, and an account executive is assigned to one or more major customers to fulfill their needs and maintain the relationship.  PURPOSE OF RELATIONSHIP MARKETING ⚫ Satisfaction Today’s customers face a growing range of choices in the products and services they can buy. They are making their choice on the basis of their perceptions of quality, service, and value. Companies need to understand the determinants of customer value and satisfaction. ⚫ Retention To create customer satisfaction, companies must manage their value chain as well as the whole value delivery system in a customer-centered way. The company’s goal is not only to get customers, but even more importantly to retain customers. Customer relationship marketing provides the key to retaining customers and involves providing financial and social benefits as well as structural ties to the customers. Companies must decide how much relationship marketing to invest in different market segments and individual customers, from such levels as basic, reactive, accountable, proactive, and full partnership  IMPORTANCE OF CRM CRM helps the organization to identify customer‘s needs and re-focus its strategy to serve them better. It helps the company to archive business growth through development edge and excellence. Some of the major issue it address are:  Identify customer needs.  Helps in rediscovering the customer and understanding him.  Identify untapped business potential.  Identify strong and weak points of supplier.  Provide feedback to the supplier on his total operation.  Provide feedback and new information on competitors.  Action plan to make organization customer – centric.  CRM CYCLE There are four phases to the customer life cycle. The four phases include; marketing, customer acquisition, relationship management, and loss. Marketing The marketing part of the customer life cycle is when messages are sent to the target market to attract prospect customers. Customer Acquisition The next phases is customer acquisition which means prospects become customers when they place an order. Relationship Management The third stage is relationship management. Relationship management is when resell processes increase the value of existing customers. Loss/Churn The end stage of a customer life cycle is loss/churn when inevitably in time a company may lose a customer. The company then needs to establish a win-back process. The company then needs to decide which lost customers  CRM CYCLE A CRM system integrates all four phases of the customer life cycle into three major processes. These processes are solicitation, lead-tracking, and relationship management. The diagram above depicts the four phases and the three major processes. It shows the flow of phases and what each phase means.  TYPES OF CRM Nowadays, three major types of customer relationship management systems, namely operational CRM, analytical CRM and collaborative CRM are being used in many organizations. 1. Operational CRM It provides support to front-office business processes that involve direct interaction with customers through any communication channel, such as phone, fax, e-mail, etc. The details of every interaction with customers, including their requirements, preferences, topics of discussion etc., are stored in the customers’ contact history and can be retrieved by the organization’s staff whenever required. Thus, it presents a unified view of customers across the organization and across all communication channels. Examples of operational CRM applications are sales force automation (SFA), customer service and support (CSS), enterprise marketing automation (EMA),etc.  2. ANALYTICAL CRM It enables to analyze customer data generated by operational CRM applications, understand the customers’ behavior, and derive their true value to the organization. This helps to approach the customers with related information and proposals that satisfy their needs. The analytical customer relationship management applications use analytical marketing tools like data mining to extract meaningful information like the buying patterns of the customers, target market, profitable and unprofitable customers, etc., that help to improve performance of the business.  3. COLLABORATIVE CRM It allows easier collaboration with customers, suppliers, and business partners and, thus, enhances sales and customer services across all the marketing channels. The major goal of collaborative customer relationship management applications is to improve the quality of services provided to the customers, thereby increasing the customers loyalty. Examples of collaborative CRM applications are partner relationship management (PRM), customer self-service and feedback, etc.  SUCCESS FACTORS IN CRM 1. Clear Objectives and Strategy 2. Customer-Centric Culture 3. Data Quality and Management 4. User Adoption and Training 5. Integration with Existing Systems 6. Personalization and Customization 7. Analytics and Insights 8. Communication and Collaboration 9. Change Management 10. Continuous Improvement 11. Executive Support and Sponsorship 12. Measurable Metrics Benefits of CRM Implementation: Improved Customer Insights: Collect and analyze data for a deeper understanding of customer behavior and preferences. Enhanced Customer Service: Streamline communication, provide personalized experiences, and resolve issues promptly. Increased Sales and Retention: Identify cross-selling and upselling opportunities, leading to revenue growth. Efficient Marketing: Targeted campaigns based on customer segments and preferences. Data-Driven Decision-Making: Access to real-time data for informed business  MIS : MANAGEMENT INFORMATION SYSTEM A Management Information System (MIS) is a computer-based system that provides information and support for managerial decision-making within an organization. It gathers, processes, stores, and disseminates information to help managers at various levels make informed decisions. MIS typically focuses on internal data and operational processes. It helps monitor the organization's performance, generate reports, and support planning and control activities.  Aspect MIS CRM ERP Sales, marketing, Departments Managers, Key Users customer service across the executives teams organization Overall Improving Streamlining Focus on organization customer business performance relationships processes Order Customer Sales reports, processing, Examples profiles, sales financial analysis inventory forecasts management Can integrate Can integrate Integrates Integration with other with other various business systems systems functions Improved Informed customer Process Benefits decision making, satisfaction, automation, cost efficiency targeted reduction  BUSINESS INTELLIGENCE ⚫ Business Intelligence (BI) is What is Business about getting the right Intelligence? information, to the right decision makers, at the right time. ⚫ BI is an enterprise-wide platform that supports reporting, analysis and decision making. ⚫ BI leads to: ⚫ fact-based decision making ⚫ “single version of the truth” What is ⚫ Making useful, actionable insight Business from stored data. Intelligenc ⚫ The act of using historical data to e gain new information. ⚫ Techniques include: ⚫ multidimensional analyses ⚫ mathematical projection ⚫ modeling ⚫ ad-hoc queries ⚫ 'canned' reporting ⚫ Dashboards  QUESTIONS BI IS DESIGNED TO ANSWER What happened? Past What is happening? Why did it happen? Presen What will happen? t What do I want to Futur happen? e Data Black ERP CRM SCM 3Pty books 5  QUESTIONS BI IS DESIGNED TO ANSWER ⚫ A BI solution, with the right data and features, should be able to take operational data and enable users to answer specific questions such as: ⚫ Sales and marketing ⚫ Which customers should I target? ⚫ What has caused the change in my pipeline? ⚫ Which are my most profitable campaigns per region? ⚫ Did store sales spike when we advertised in the local paper or launched an email campaign? ⚫ What is the most profitable source of sales leads and how has that changed over time? ⚫ Operational ⚫ Which vendors are best at delivering on time and on budget?– How many additional personnel do we need to add per store during the holidays? ⚫ Which order processing processes are most inefficient? ⚫ Financial ⚫ What is the fully loaded cost of new products? ⚫ What is the expected annual profit/loss based on current marketing and sales forecasts? ⚫ How are forecasts trending against the annual plan? ⚫ What are the current trends in cash flow, accounts payable and accounts receivable and how do they compare with plan? ⚫ Overall business performance ⚫ What are the most important risk factors impacting the company’s ability to meet annual profit goals? ⚫ Should we expand internationally and, if so, which geographic areas should we  BUSINESS INTELLIGENCE VISION Improving organizations by providing business insights to all employees leading to better, faster, more relevant decisions. Advanced Analytics Self Service Reporting End-User Analysis Business Performance Management Operational 8  Business Analytics vs Business Intelligence Aspect Business Analytics Business Intelligence Data collection, integration, and Data analysis to gain insights into past Focus presentation for informed decision- performance and predict future trends. making. Provide historical, current, and Extract actionable insights for strategic Objective predictive views of business and operational decisions. operations. Deeper analysis of specific problems Broader overview of organizational Scope using statistical and quantitative data for reporting and monitoring. methods. Analyzes raw data to discover patterns, Organizes data into understandable Data Usage correlations, and causality. formats like dashboards and reports. Advanced statistical modeling, predictive Reporting, data visualization, OLAP, Techniques modeling, data mining. data warehousing. Future-oriented, emphasizes predictive Past and present-oriented, emphasizes Time Focus analysis. historical reporting. Managers, executives, and operational Business Analytics (BA): Focuses on advanced statistical analysis and predictive modeling. Aims to extract actionable insights from data to drive decision-making. Often involves analyzing specific problems, uncovering patterns, and making predictions. Typically used by data scientists and analysts to forecast trends and outcomes. Business Intelligence (BI): Emphasizes data presentation and reporting for a broader organizational view. Provides historical, current, and predictive views of business operations. Used to monitor and visualize key performance indicators (KPIs) and track trends over time. Designed for managers, executives, and operational staff to support informed decision- making.  Indicative use of Business Analytics and Business Intelligence by the few known companies Company Business Analytics (BA) Business Intelligence (BI) - Analyzing viewer behavior patterns to - Providing executive dashboards with real-time Netflix personalize content recommendations. viewership metrics. - Optimizing pricing strategies using real- - Analyzing historical sales data to forecast Amazon time data to adjust prices based on demand, manage inventory, and optimize supply demand and competition. chain. - Utilizing dynamic pricing models based - Monitoring driver efficiency, ride patterns, and Uber on factors like real-time demand, traffic, customer feedback through interactive and weather conditions. dashboards. - Analyzing historical sales data and - Generating financial reports, balance sheets, Coca-Cola consumer trends to forecast demand and sales performance summaries for and adjust production accordingly. management and investors. - Applying demand forecasting models to - Creating dashboards for store managers to Walmart optimize inventory levels and prevent track sales, inventory turnover, and employee stockouts. performance. - Analyzing customer preferences and - Providing regional managers with insights into Starbucks purchase patterns to introduce new real-time sales, customer preferences, and store products and tailor menus. performance.  Examples of BI Microsoft BI 10  BUSINESS INTELLIGENCE USERS 4 Types of ⚫ Executives : Information is summarized and been has defined for them. Users have the Users view static information online and/or print to a ability printer.to local ⚫ Casual Users Casual users require the next level of detail the from information that is provided to addition viewers. to the privileges of a viewer, casual In have users the ability to refresh report information ability and to the enter desired information the purposes parameters for performing high-level research of analysis. and ⚫ Functional Users Functional users need to perform detailed and analysis, which requires access research transactional to data. In addition to the privileges casual of a user, functional users have the ability develop to their own ad hoc queries and OLAP analysis. perform ⚫ Super Users Super users have a strong understanding of the both business and technology to access and transactional analyze data. They have full privileges explore to and analyze the data with the applications BI available to them. Business Analytics Software Business Intelligence Software RStudio Tableau Python (pandas, NumPy, scikit- Microsoft Power BI learn) MATLAB QlikView/Qlik Sense IBM SPSS MicroStrategy RapidMiner IBM Cognos Analytics KNIME Google Data Studio Alteryx SAP BusinessObjects SAS Analytics Domo TIBCO Statistica Looker Weka Sisense Orange Dundas BI D3.js (JavaScript data visualization) Pentaho Microsoft Excel (data analysis Yellowfin BI features) H2O.ai Zoho Analytics  What are the differences between... OL TP VS OL AP Why would an organization need an OLAP? Ask me what's Ask me what will happeing happen OLTP OLAP OLTP OLAP On­Line Transaction Processing On­Line Analytical Processing OLTP, or Online Transaction Processing, is a type of database management system and processing methodology designed for managing and handling transaction-oriented workloads in real-time. It is primarily used in scenarios where many users need concurrent access to a database, and the focus is on quick and efficient processing of individual transactions. Characteristics include: 1.Transaction-oriented 2.Real-time processing 3.High concurrency 4.Data integrity (ACID properties) 5.Normalized data 6.Indexing 7.Small, frequent reads/writes 8.Relational Database Management Systems (RDBMS) 9.Reporting and analytics separation Examples: OLTP systems are commonly used in various industries, including e-commerce (e.g., online shopping carts), banking (e.g., ATM transactions), healthcare (e.g., patient record management), and more. OL T OLTP's are operational systems which help execute and P record the day to day operations of a business Houses the original source of the data Control and Run fundamental business tasks Snapshot of on-going business process OLTP Constant modifications to data input by end users Simple queries performed to yield non-complex records Quick processing speeds to keep up with daily transactions Highly normalized with many tables OL T P With all of the information and transactions the OLTP system is required to handle, utilizing the system to run complex queries can put an extreme an unnecessary load onto the system. (This can in turn cause detrimental lags in the daily 'transaction' functionality, which the OLTP is designed to run) Back ups must be done frequently as data loss in your OLTP can lead to Monetary and Legal complications. Aspect Advantages Limitations - High-speed processing of small - May not handle complex queries Performance transactions efficiently - Potential for locking and contention Concurrency - Supports multiple concurrent users issues Data Integrity - Maintains ACID properties - Increased overhead for data integrity - Provides real-time or near-real-time Real-time - Scalability challenges with high loads data Normalized Data - Reduces data redundancy - Complex queries may require joins Indexing - Optimizes data retrieval - Index maintenance overhead - May not suit batch processing Small Transactions - Efficient handling of small operations scenarios - Suitable for e-commerce, banking, Examples - Less suitable for data warehousing healthcare - Supported by many relational - May require complex schema RDBMS Support databases management OLAP, or Online Analytical Processing, is a category of computer processing that facilitates complex, multidimensional analysis of large volumes of data. It's primarily used for business intelligence and decision support systems. Overview of OLAP (Online Analytical Processing) in list format: 1. Multidimensional Data Model 2. Data Aggregation 3. Complex Queries 4. Speed of Query Response 5. Read-Only Data 6. Data Warehouses 7. Star and Snowflake Schema 8. Reporting and Visualization 9. Types of OLAP (MOLAP, ROLAP, HOLAP) 10.Scalability 11.Security Examples: OLAP is used in various domains, including finance (for financial analysis and reporting), retail (for sales and inventory analysis), and healthcare (for clinical data analysis). OL A P OLAP's are data warehouses which leverage data acquired from OLTP systems to help organzations make better decisions Consolidation of data from OLTP system(s) Helps with Planning, Problem Solving, and making Decision Multi_Dimensional views of business activities OLAP Periodic Refreshes made to upload data from OLTP system(s) Allows for the execution of complex queries (aggregations) Processing speed can vary depending on system structure More often de-normalized with fewer tables, cubes utilized OL A P Olap's are comprised of all necessary data from your OLTP system, where it is structured and aggregated to allow for detail intensive queries to be performed. Highly visual reports can be extracted and presented in a manner fit for end users A great benefit of an olap system is the potential for the system to be utilized as a 'back­up' of the oltp by regularly reloading the data from the oltp to olap. Aspect Advantages Limitations Multidimensional Model - Intuitive data representation - Complex data modeling - Pre-aggregation might lead to data Data Aggregation - Flexible summarization of data loss - Supports intricate analytical - Slower query response with large Complex Queries queries datasets - Rapid response to analytical - Resource-intensive for complex Query Speed queries queries - Limited for transactional Read-Only Data - Data integrity is maintained processing - Requires ETL (Extract, Transform, Data Warehouses - Centralized, historical data storage Load) - Star and snowflake schemas Schema Types - Schema design can be complex simplify modeling - Tools may require training and Reporting and Visualization - Enables effective data presentation integration - Initial setup and maintenance can Examples in Various Domains - Wide range of applications be costly - Allows choice of OLAP system - Different types have varying  OL T OL A P VS P Complimen tary OLTP OLAP While it is very important to ensure you are collecting data as efficiently as possible, your ability to leverage that data to make decisions ensures operational longevity and success. The data created by your OLTP can be seen as 'fuel' for your enterprise, where an OLTP system is the vehicle which filters, extracts, and consumes that fuel to analyze, predict and propel best business practices. Types of OLAP – ROLAP, MOLAP, HOLAP There are three different types of OLAP based on how data is stored in the database. ROLAP : stands for Relational Online Analytical Processing. It stores data in the form of rows and columns. ROLAP does not pre-compute data; it can be accessed through SQL queries on demand. Thus, ROLAP empowers users to analyze and view data, and is capable of saving storage space while working with massive historical datasets that are not often queried. It can deal with large datasets, but the larger is the dataset more is the processing time. Thus, performance becomes an issue with rising data volumes and concurrencies. MOLAP: MOLAP is an acronym for Multidimensional Online Analytical Processing. In MOLAP, data is pre-aggregated, summarized, and stored in the form of a multidimensional array. It enables users to model data and visualize it from multiple viewpoints. Since all the complex calculations are done in advance, users can easily perform slice and dice operations on their data with fast response times. However, traditional MOLAP is less scalable than ROLAP, as a limited amount of data can be stored in a multidimensional cube. HOLAP : Hybrid OLAP is a combination of both MOLAP and ROLAP features. It uses both relational and multidimensional structures to store data, and which one should be used to access data depends on the processing application. Thus, HOLAP provides a mid-way approach to both the methods described above. WHAT IS CRISP-DM? Cross Industry Standard Process for Data Mining Developed in 1996 by big Business Understanding Data Understanding players in data analysis 01001110010101 011100100111000110 Data SPSS, Teradata, Daimler, OCHRA, 100101110101 1000D10A01T11A01001 Preparation NCR Deployment 10000000111000001 10000110110110 for data-centric projects 110000110010010001 Most popular methodology Modelling It can be regarded as agile Evaluation Introduces almost no overhead Emphasizes adaptive transitions between project phases  BUSINESS UNDERSTANDING Determine business objectives Business Data Assess situation Understanding Understanding Resources (data!), risks, costs & benefits 01001110010101 011100100111000110 Data 100101110101 1000D10A01T11A01001 Preparation 10000000111000001 Deployment 10000110110110 Determine data mining goals 110000110010010001 Ideally with quantitative success Modelling criteria Evaluation Develop project plan Estimate time line, budget, but also tools and techniques  BUSINESS UNDERSTANDING Determining Business Objectives 1. Gather background information  Compiling the business background  Defining business objectives  Business success criteria 2. Assessing the situation  Resource Inventory  Requirements, Assumptions, and Constraints  Risks and Contingencies  Cost/Benefit Analysis 4. Determining data science goals  Data science goals  Data science success criteria 4. Producing a Project Plan  EXAMPLE OF THE PROJECT PLAN Phase Time Resources Risks Business 1 week All analysts Economic change understanding Data 3 weeks All analysts Data problems, technology understanding problems Data preparation 5 weeks Data scientists, DB Data problems, technology engineers problems Modeling 2 weeks Data scientists Technology problems, inability to build adequate model Evaluation 1 week All analysts Economic change, inability to implement results Deployment 1 week Data scientist, DB Economic change, inability engineers, to implement results implementation team  BUSINESS UNDERSTANDING Difficult! Often, you have to enter a new field You have to explain data science limitations to non- experts No, performance will not be 100% We need much more data to train an accurate model For tomorrow, it is impossible Source: http://xkcd.com/1425  BUSINESS UNDERSTANDING – DOS AND DON'TS Have a lot of patience for vaguely defined problems Learn to concretize or even reduce the scope of the initial idea Data sample Real-life use cases Quantitative success metrics Do not waste your time on ill-defined, unrealistic projects  READY FOR THE DATA UNDERSTANDING? From a business perspective:  What does your business hope to gain from this project?  How will you define the successful completion of our efforts?  Do you have the budget and resources needed to reach our goals?  Do you have access to all the data needed for this project?  Have you and your team discussed the risks and contingencies associated with this project?  Do the results of your cost/benefit analysis make this project worthwhile? From a data science perspective:  How specifically can data mining help you meet your business goals?  Do you have an idea about which data mining techniques might produce the best results?  How will you know when your results are accurate or effective enough? (Have we set a measurement of data mining success?)  How will the modeling results be deployed? Have you considered deployment in your project plan?  Does the project plan include all phases of CRISP-DM?  Are risks and dependencies called out in the plan?  DATA UNDERSTANDING 1. Collect initial data  Existing data Business Data  Purchased data Understanding Understanding  Additional data 01001110010101 Data 2. Describe data 011100100111000110 100101110101 Preparation 1000D10A01T11A01001  Amount of data 10000000111000001 Deployment 10000110110110  Value types 110000110010010001  Coding schemes Modelling 3. Explore data 4. Verify data quality Evaluation  Missing data  Data errors  Coding inconsistencies  Bad metadata  DATA UNDERSTANDING –DOS AND DON'TS Do not economize on this phase The earlier you discover issues with your data the better (yes, your data will have issues!) Data understanding leads to domain understanding, it will pay off in the modelling phase Do not trust data quality estimates provided by your customer Verify as far as you can, if your data is correct, complete, coherent, deduplicated, representative, independent, up-to-date, stationary… Investigate what sort of processing was applied to the raw data Understand anomalies and outliers  READY FOR THE DATA PREPARATION?  Are all data sources clearly identified and accessed? Are you aware of any problems or restrictions?  Have you identified key attributes from the available data?  Did these attributes help you to formulate hypotheses?  Have you noted the size of all data sources?  Are you able to use a subset of data where appropriate?  Have you computed basic statistics for each attribute of interest? Did meaningful information emerge?  Did you use exploratory graphics to gain further insight into key attributes? Did this insight reshape any of your hypotheses?  What are the data quality issues for this project? Do you have a plan to address these issues?  Are the data preparation steps clear? For instance, do you know which data sources to merge and which attributes to filter or select?  DATA PREPARATION 1. Select right data Business Data Understanding Understanding  Select training examples  Select features 01001110010101 011100100111000110 Data 2. Clean data 100101110101 1000D10A01T11A01001 Preparation 10000000111000001  Fill in missed data Deployment 10000110110110 110000110010010001  Correct data errors  Make coding consistent Modelling 3. Extend data  Extend training examples Evaluation  Extend features 4. Format data  Put data in a format for training the model  Tedious! classification-jsonl metadata-jsonl metadata-extracted-jsonl projects-from-iis-jsonl projects-from-infspace-jsonl data-aux/class-riffle data-aux/metad-riffle data-aux/priis-json data-aux/prinf-json data-clean/joind-jsonl stat/basic stat/basic-fp7 stat/collab Use workflow tools to document, automate & parallelize data prep.  Data understanding and preparation will usually consume half or more of your project time! What % of time in your data mining project(s) is spent on data cleaning and preparation? 8% Percentage of time 4% 25% 20% 20% 14% 10% 10% 10% 39% 25% Percentage of responses DATA PREPARATION – DOS AND DON'TS Automate this phase as far as possible When merging multiple sources, track provenance of your data Use workflow tools to help you with the above Prepare your customer that data understanding and preparation take considerable amount of time  READY FOR THE MODELING?  Based upon your initial exploration and understanding, were you able to select relevant subsets of data?  Have you cleaned the data effectively or removed unsalvageable items? Document any decisions in the final report.  Are multiple data sets integrated properly? Were there any merging problems that should be documented?  Have you researched the requirements of the modeling tools that you plan to use?  Are there any formatting issues you can address before modeling? This includes both required formatting concerns as well as tasks that may reduce modeling time.  MODELLING Select modelling technique Assumptions, measure of accuracy Business Data Understanding Understanding Generate test 01001110010101 Data design 011100100111000110 100101110101 Preparation 1000D10A01T11A01001 10000000111000001 Deployment 10000110110110 110000110010010001 Build model Modelling Feature eng., optimize model parameters Evaluation Assess model Iterate the above  MODELLING – TOOLING SELECTION Should I use general purpose language? Breadth = performance, lots of general purpose libraries and tooling, easy creation of web services R (quality of data analysis tooling) Matlab Should I use data analysis Mathematica language? Depth = easy data Python Depth manipulation, latest models and Scala statistical techniques available F# Clojure Where your model will be deployed? Java C++ C# Do you need to distribute your computations? (avoid!) Breadth (quality of general purpose tooling) Can I afford a prototype?  MODELLING – MY DOS AND DON'TS Be creative with your features (feature engineering) Esp. from textual data or time-series you can generate a lot of std. features Make conscious decision about missing data (NAs) and outliers (regression!) Allocate time for hyperparameter optimization Whenever possible, peek inside your model and consult it with domain Assess expert feature importance Run your model on simulated data Develop your model with deployment conditions in mind  READY FOR THE EVALUATION?  Are you able to understand the results of the models?  Do the model results make sense to you from a purely logical perspective? Are there apparent inconsistencies that need further exploration?  From your initial glance, do the results seem to address your organization’s business question?  Have you used analysis nodes and lift or gains charts to compare and evaluate model accuracy?  Have you explored more than one type of model and compared the results?  Are the results of your model deployable?  EVALUATION 1. Evaluate the results  Are results presented clearly?  Are there any novel findings? Business Data  Can models and findings be applicable to business Understanding Understanding goals?  How well do the models and findings answer business 01001110010101 011100100111000110 Data 100101110101 goals? 1000D10A01T11A01001 Preparation 10000000111000001  What additional questions the modeling results have Deployment 10000110110110 110000110010010001 risen? Modelling 2. Review the process  Did the stage contribute to the value of the results?  What went wrong and how it can be fixed? Evaluation  Are there alternative decisions which could have been executed? 3. Determine the next steps  EVALUATION –DOS AND DON'TS WORK WITH THE PERFORMANCE CRITERIA DICTATED BY YOUR CUSTOMER'S BUSINESS MODEL ASSESS NOT ONLY PERFORMANCE, BUT ALSO PRACTICAL ASPECTS, RELATED TO DEPLOYMENT, FOR EXAMPLE: Training and prediction speed Robustness and maintainability (tooling, dependence on other subsystems, library vs. homegrown code) Watch out for data leakage, for example: Time series – mixing past and future Meaningful identifiers Other nasty ways of artificially introducing extra information, not available in production  DEPLOYME NT Plan deployment Business Data Understanding Understanding Plan monitoring and maintenance 01001110010101 011100100111000110 Data 100101110101 1000D10A01T11A01001 Preparation 10000000111000001 Deployment 10000110110110 110000110010010001 Produce final Modelling report Evaluation Review project Collect lessons learned!  WHAT IS KPI? Definition of 'Key Performance Indicators - KPI‘ A set of quantifiable measures that a company or industry uses to gauge or compare performance in terms of meeting their strategic and operational goals. KPIs vary between companies and industries, depending on their priorities or performance criteria. Also referred to as "key success indicators (KSI)".  OBJECTIVES OF KPI Improve personnel’s understanding of KPIs. + = Improve personnel’s awareness of maintenance performance.  KPIs are directly linked to the overall goals of the company. KPIs are measurements that define and track specific business goals and objectives. Key Success Key Business Performance Factors Determin Tracked Indicators (KSFs) by. Objective e. (KPIs) s ⚫The larger or smaller organizational strategies require monitoring, improvement, and evaluation. ⚫Once an organization has analyzed its mission, identified all its stakeholders, and defined its goals, it needs a way to measure progress toward those goals. ⚫KPIs are utilized to track or measure actual performance against key success factors.  Key Success Factors (KSFs) only change if there is a fundamental shift in business objectives. Key Performance Indicators (KPIs) change as objectives are met, or management focus shifts.  WHY USE KPI’S? ⚫Performance effectiveness. ⚫For the accuracy, actual reflection of the process, efficacy in delivering the outcome. ⚫The effects of a change can be monitored reliably, repeatedly and accurately by KPI. ⚫A KPI can be used to closely monitor the results of actions. ⚫Detect potential problems and it can drive improvement. ⚫It is reasonableto use the KPI as a tool to improve ongoing process performance.  USES OF KPI ⚫ A key performance indicator (KPI) or performance indicator is used to periodically measure and assess the performances of organizations, business units, and their division, departments and employees. ⚫ To make the decision making process easier. ⚫ It help organizations to understand how well they are performing in relation to their strategic goals and objectives, i.e. helps in Promoting Accountability. ⚫ They are used by an organization to evaluate its success or the success of a particular activity in the organization. ⚫ To analyze the operational details of the organization. ⚫ It helps to focus on the facts clearly. ⚫ It helps in creating a Culture of Learning.  HOW TO DESIGN KPI’S ⚫KPIs should be clearly linked to the strategy, i.e. the things that matter the most. ⚫KPIs have to provide the answers to our most important questions. ⚫KPIs should be primarily designed to employees and provide them with empower information to learn. the relevant  IDENTIFYING THE KPI’S ⚫Related to strategic aims. ⚫Identify what makes the organization success or failures. ⚫Controllable and accountable. ⚫Qualitative and quantitative. ⚫Long term and short term. ⚫Consider Stakeholder needs. ⚫Identify important aspects. Then, Establish Company Goals and KPIs. Select Performance Indicators and Metrics. Set Targets and Track Performance.  TYPES OF KPI Quantitative indicators: Quantitative indicators are represented by continuous or discrete numbers, which can be ratios, percentages, or whole numbers that represent values like rating scales, dollars, or weight. These indicators are the most straightforward quantifiable measures of performance, as they present direct numerical values. Qualitative indicators: These indicators are not expressed numerically but through feelings or opinions. An employee satisfaction survey can be an example of qualitative data where performance is based on feedback. Leading indicators: Leading indicators are variables that can help identify long-term trends and possibly predict successful future outcomes of your business processes. Lagging indicators: Lagging KPIs compare a business’ current performance in a particular field with their past performance in the same field. Input indicators: Input indicators are a type of KPI that track the resources necessary to produce the intended outcome, such as funding or extra staff. Input indicators can help companies keep track of how efficiently they are using their resources. Output indicators: Output indicators measure the success or failure of your business activities, like the number of goods or services created through a particular process. Revenue growth and new customer acquisition also indicate how well your business is performing. Process indicators: Process indicators represent the efficiency of a business’s process and how effectively it is functioning. Practical indicators: Practical indicators explore the function of an existing process at a company, usually involving observation or feedback on that process. Directional indicators: Directional indicators help determine the company’s success in comparison with competitors, while practical indicators are specific to the company’s process within itself. Actionable indicators: Actionable KPIs measure a company’s ability to enact change whether through political action or a shift in company culture. Financial indicators: Financial indicators are a marker of a business’s monetary growth and stability. When paired with other KPIs, this indicator can help paint a more complete picture of your company’s financial viability. Outcome indicators: These indicators are a marker of whether the program is meeting its goals via the short or long term.  CHARACTERISTICS OF A GOOD KPI ⚫KPI is always connected with the corporate goals. ⚫A ⚫. KPI are decided by the management ⚫They are the leading indicators of desired performance by the organization. ⚫Easy to understand A KPI need to be: ⚫ Specific ⚫ Measurable ⚫ Achievable ⚫ Result-oriented or Relevant ⚫ Time-bound  SHIPPING AND LOGISTICS The main five KPI’S in shipping and logistic industries are: ⚫ Sales forecasts. ⚫ Inventory. ⚫ Procurement and suppliers. ⚫ Warehousing. ⚫ Transportation.  INFRASTRUCTURE SECTOR The main five KPI’s in Infrastructure sector are: ⚫ Client Satisfaction. ⚫ Construction Time & Cost. ⚫ Productivity. ⚫ Defects. ⚫ Profitability.