Binus University ISYS6198003 Session 15&16 Physical Database Design PDF

Course : ISYS6198003 – Data and Information Management Year : 2024 Methodology – Physical Database Design for Relational Databases Session 15&16  LEARNING OUTCOMES LO-3 : Design a database that includes conceptual, logical, and physical based database de...

Course : ISYS6198003 – Data and Information Management Year : 2024 Methodology – Physical Database Design for Relational Databases Session 15&16  LEARNING OUTCOMES LO-3 : Design a database that includes conceptual, logical, and physical based database design methodology LO-4 : Apply Database using Structured Data Model which fits the Artificial Intelligence workflow  ACKNOWLEDGEMENT These slides have been adapted from: - Thomas Connolly and Carolyn Begg. 2015. Database Systems: A Practical Approach To Design, Implementation, and Management. Pearson Education. USA. ISBN:978-1- 292-06118-4, Chapter 18 - Oracle Academy, Database Design, Section 9  LEARNING OBJECTIVES Purpose of physical database design. How to map the logical database design to a physical database design. How to design base relations for target DBMS. How to design general constraints for target DBMS. How to select appropriate file organizations based on analysis of transactions. When to use secondary indexes to improve performance. How to estimate the size of the database. How to design user views. How to design security mechanisms to satisfy user requirements. Datasets for Machine Learning in Artificial Intelligence workflows Bina Nusantara METHODOLOGY – PHYSICAL DATABASE DESIGN FOR RELATIONAL DATABASES  Logical vs Physical Database Design Sources of information for physical design process includes logical data model and documentation that describes model. Logical database design is concerned with the what, physical database design is concerned with the how. Bina Nusantara  Terminology Mapping Changing from analysis (conceptual and logical model) to implementation (physical model) also means changing terminology: An entity becomes a table. An instance becomes a row. An attribute becomes a column. A primary unique identifier becomes a primary key. A secondary unique identifier becomes a unique key. A relationship is transformed into a foreign-key column and a foreign key constraint. Bina Nusantara  Physical Database Design Process of producing a description of the implementation of the database on secondary storage. It describes the base relations, file organizations, and indexes used to achieve efficient access to the data, and any associated integrity constraints and security measures. Bina Nusantara  Overview of Physical Database Design Methodology Step 3 Translate logical data model for target DBMS – Step 3.1 Design base relations – Step 3.2 Design representation of derived data – Step 3.3 Design general constraints Step 4 Design file organization and indexes – Step 4.1 Analyze Transactions – Step 4.2 Choose file organizations – Step 4.3 Choose indexes – Step 4.4 Estimate disk space requirements Bina Nusantara  Overview of Physical Database Design Methodology Step 5 Design user views Step 6 Design security mechanisms Step 7 Consider the introduction of controlled redundancy Step 8 Monitor and tune operational system Bina Nusantara  Step 3 Translate Logical Data Model for Target DBMS To produce a relational database schema from the logical data model that can be implemented in the target DBMS. Need to know functionality of target DBMS such as how to create base relations and whether the system supports the definition of: – PKs, FKs, and AKs; – required data – i.e. whether system supports NOT NULL; – domains; – relational integrity constraints; – general constraints. Bina Nusantara  Step 3.1 Design base relations To decide how to represent base relations identified in logical model in target DBMS. For each relation, need to define: – the name of the relation; – a list of simple attributes in brackets; – the PK and, where appropriate, AKs and FKs. – referential integrity constraints for any FKs identified. Bina Nusantara  Step 3.1 Design base relations From data dictionary, we have for each attribute: – its domain, consisting of a data type, length, and any constraints on the domain; – an optional default value for the attribute; – whether it can hold nulls; – whether it is derived, and if so, how it should be computed. Bina Nusantara  DBDL for the PropertyFo rRent Relation Bina Nusantara  Example 1 –DBDL with Super/subtypes CREATE TABLE payment ( payment_id NUMBER NOT NULL, order_order_id NUMBER NOT NULL, amount DECIMAL, PRIMARY KEY ( payment_id ), FOREIGN KEY ( order_order_id ) REFERENCES "Order" ( order_id )); CREATE TABLE cash ( payment_payment_id NUMBER NOT NULL, PRIMARY KEY ( payment_payment_id ), FOREIGN KEY ( payment_payment_id ) REFERENCES payment ( payment_id CREATE TABLE creditcard ( payment_payment_id NUMBER NOT NULL, nama VARCHAR2(50), nomor NUMBER, expireddate DATE, bank_bank_id NUMBER NOT NULL, PRIMARY KEY ( payment_payment_id ), FOREIGN KEY ( bank_bank_id ) REFERENCES bank ( bank_id ), FOREIGN KEY ( payment_payment_id ) REFERENCES payment ( payment_id  Example 2 – DBDL with Arcs CREATE TABLE in_house ( in_house_id NUMBER NOT NULL, PRIMARY KEY ( in_house_id ) CREATE TABLE external (external_id NUMBER NOT NULL, PRIMARY KEY ( external_id )); CREATE TABLE training ( training_id INTEGER NOT NULL, training_name VARCHAR2(30) NOT NULL, external_external_id NUMBER, in_house_in_house_id NUMBER,' PRIMARY KEY ( training_id ), FOREIGN KEY ( external_external_id ) REFERENCES external ( external_id ), FOREIGN KEY ( in_house_in_house_id ) REFERENCES in_house ( in_house_id )); ALTER TABLE training ADD CONSTRAINT fkarc_4 CHECK ( ( ( in_house_in_house_id IS NOT NULL ) AND ( external_external_id IS NULL ) ) OR ( ( external_external_id IS NOT NULL ) AND ( in_house_in_house_id IS NULL ) ) OR ( ( in_house_in_house_id IS NULL ) AND ( external_external_id IS NULL ) ) );  Step 3.2 Design representation of derived data To decide how to represent any derived data present in logical data model in target DBMS. Examine logical data model and data dictionary, and produce list of all derived attributes. Derived attribute can be stored in database or calculated every time it is needed. Bina Nusantara  Step 3.2 Design representation of derived data Option selected is based on: additional cost to store the derived data and keep it consistent with operational data from which it is derived; cost to calculate it each time it is required. Less expensive option is chosen subject to performance constraints. Bina Nusantara  PropertyforRent Relation and Staff Relation with Derived Attribute noOfProperties SELECT s.staffNo, fName,lName, s.branchNo, COUNT(propertyNo) FROM Staff s JOIN PropertyForRent p ON s.staffNo=p.staffNo GROUP BY s.staffNo, fName,lName, s.branchNo SELECT staffNo, COUNT(propertyNo) FROM PropertyForRent GROUP BY staffNo Bina Nusantara  Step 3.3 Design general constraints To design the general constraints for target DBMS. Some DBMS provide more facilities than others for defining enterprise constraints. Example: CONSTRAINT StaffNotHandlingTooMuch CHECK (NOT EXISTS (SELECT staffNo FROM PropertyForRent GROUP BY staffNo HAVING COUNT(*) > 100)) Bina Nusantara  Step 4 Design File Organizations and Indexes To determine optimal file organizations to store the base relations and the indexes that are required to achieve acceptable performance; that is, the way in which relations and tuples will be held on secondary storage. Must understand the typical workload that database must support. Bina Nusantara  Step 4.1 Analyze transactions To understand the functionality of the transactions that will run on the database and to analyze the important transactions. Attempt to identify performance criteria, such as: – transactions that run frequently and will have a significant impact on performance; – transactions that are critical to the business; – times during the day/week when there will be a high demand made on the database (called the peak load). Bina Nusantara  Step 4.1 Analyze transactions Use this information to identify the parts of the database that may cause performance problems. Also need to know high-level functionality of the transactions, such as: – attributes that are updated; – search criteria used in a query. Often not possible to analyze all transactions, so investigate most ‘important’ ones. To help identify these can use:  transaction/relation cross-reference matrix, showing relations that each transaction accesses, and/or  transaction usage map, indicating which relations are potentially heavily used. Bina Nusantara  Step 4.1 Analyze transactions To focus on areas that may be problematic: (1) Map all transaction paths to relations. (2) Determine which relations are most frequently accessed by transactions. (3) Analyze the data usage of selected transactions that involve these relations. Bina Nusantara  Cross-referencing transactions and relations Bina Nusantara  Example Transaction Usage Map Bina Nusantara  Example Transacti on Analysis Form Bina Nusantara  Step 4.2 Choose file organizations To determine an efficient file organization for each base relation. File organizations include Heap, Hash, Indexed Sequential Access Method (ISAM), B+-Tree, and Clusters. Some DBMSs may not allow selection of file organizations. Bina Nusantara  Step 4.3 Choose indexes To determine whether adding indexes will improve the performance of the system. One approach is to keep tuples unordered and create as many secondary indexes as necessary. www.progress.com Bina Nusantara  Step 4.3 Choose indexes Another approach is to order tuples in the relation by specifying a primary or clustering index. In this case, choose the attribute for ordering or clustering the tuples as: – attribute that is used most often for join operations - this makes join operation more efficient, or – attribute that is used most often to access the tuples in a relation in order of that attribute. Bina Nusantara  Step 4.3 Choose indexes If ordering attribute chosen is key of relation, index will be a primary index; otherwise, index will be a clustering index. Each relation can only have either a primary index or a clustering index. Secondary indexes provide a mechanism for specifying an additional key for a base relation that can be used to retrieve data more efficiently. Bina Nusantara  Step 4.3 Choose indexes Have to balance overhead involved in maintenance and use of secondary indexes against performance improvement gained when retrieving data. This includes: – adding an index record to every secondary index whenever tuple is inserted; – updating secondary index when corresponding tuple updated; – increase in disk space needed to store secondary index; – possible performance degradation during query optimization to consider all secondary indexes. Bina Nusantara  Step 4.3 Choose indexes – Guidelines for choosing ‘wish- list’ 1. Do not index small relations. 2. Index PK of a relation if it is not a key of the file organization. 3. Add secondary index to a FK if it is frequently accessed. 4. Add secondary index to any attribute heavily used as a secondary key. 5. Add secondary index on attributes involved in: selection or join criteria; ORDER BY; GROUP BY; and other operations involving sorting (such as UNION or DISTINCT). Bina Nusantara  Step 4.3 Choose indexes – Guidelines for choosing ‘wish- list’ 6. Add secondary index on attributes involved in built-in functions. 7. Add secondary index on attributes that could result in an index-only plan. 8. Avoid indexing an attribute or relation that is frequently updated. 9. Avoid indexing an attribute if the query will retrieve a significant proportion of the relation. 10. Avoid indexing attributes that consist of long character strings. Bina Nusantara  Step 4.4 Estimate disk space requirements To estimate the amount of disk space that will be required by the database. www.partition-tool.com Bina Nusantara  Step 5 Design User Views To design the user views that were identified during the Requirements Collection and Analysis stage of the database system development lifecycle. Example : Branch, consisting of the Director and Manager User Views; StaffClient, consisting of the Supervisor, Assistant, and Client User Views Bina Nusantara  Step 6 Design Security Measures To design the security measures for the database as specified by the users. Bina Nusantara www.valiantsolutions.com Datasets for Machine Learning  Machine Learning Open Datasets Your machine learning program is only as good as your training sets. Data sets are an integral part of the quality of your machine learning. Natural Language Processing  Amazon Reviews: A collection of over 35 million reviews from the last 18 years. It includes things like ratings, reviews in plain text, and user information. It also contains complete product information for reference.  Wikipedia Links Data: The full power of Wikipedia including four million articles containing 1.9 billion words. Your search options are varied and include both word and phrase searches as well as pieces of paragraphs. Sentiment Analysis  – Standford Sentiment Treebank: Dataset containing sentiment notations for over 10,000 pieces of data from Rotten Tomatoes reviews rendered in HTML  – Twitter US Airline Sentiment: Tweets collected about US Airlines with clear markers for positive, negative, and neutral tones, dated from 2015. https://opendatascience.com/25-excellent-machine-learning-open-datasets/  Machine Learning Open Datasets Facial Recognition  Labeled Faces In The Wild: Common dataset for facial recognition training. It includes 13,000 cropped faces plus a subset of people with two different pictures within the dataset.  UMDFaces Dataset: Includes both still and video images. The dataset is annotated and features around 367,000 faces of over 8,000 subjects. Image Datasets  Imagenet: Dataset containing over 14 million images available for download in different formats. It also includes API integration and is organized according to the WordNet hierarchy.  Google’s Open Images: 9 million URLs to categorized public images in over 6,000 categories. Each image is licensed under creative commons. Health:  – FiveThirtyEight Journalism: The numbers behind some of this journalism hub’s stories. Useful for visualizations and data stories.  – BuzzFeed Media: Open source data hub for everything in the realm of Buzzfeed. Everything their journalists used to produce the stories (the organization recommends reading the articles to get a better https://opendatascience.com/25-excellent-machine-learning-open-datasets/ idea of how the data was used.)  Machine Learning Open Datasets Dataset Aggregators – OpenDataSoft: 2600 data portals arranged in an interactive map formation or by country list. If you’re looking for it, chances are, it’s here. – Kaggle: an online community of data scientists where users can work with and upload datasets. It’s a community and a resource in one. – UCI Machine Learning Repository: User contributed datasets in various levels of cleanliness. It’s one of the originals, and you can download datasets without having to register anything. https://opendatascience.com/25-excellent-machine-learning-open-datasets/  Exercise Based on your group project, create physical database design (step 3.1 until step 3.3)  SCHOOL OF INFORMATION SYSTEMS http:// sis.binus.ac.id/ http ://www.facebook.com/schoolisbinus SIS Binus