CS403 Database Management Systems Handouts PDF

Database Management System (CS403) Database Management System (CS403) VU DATABASE MANAGEMENT SYSTEM......................................................................................... 1 L...

Database Management System (CS403) Database Management System (CS403) VU DATABASE MANAGEMENT SYSTEM......................................................................................... 1 LECTURE NO. 01........................................................................................................................... 9 Reading Material..................................................................................................................... 9 Overview of Lecture................................................................................................................ 9 Introduction to the course................................................................................................... 9 Database definitions:........................................................................................................ 10 Importance of the Databases........................................................................................... 12 Databases and Traditional File Processing Systems....................................................... 12 Advantages of Databases................................................................................................ 15 LECTURE NO. 02......................................................................................................................... 17 Reading Material................................................................................................................... 17 Overview of Lecture.............................................................................................................. 17 Difference between Data and Information........................................................................ 17 Further Advantages of Database Systems:...................................................................... 19 Cost Involved:................................................................................................................... 21 Importance of Data........................................................................................................... 22 Levels of Data................................................................................................................... 22 Users of Database Systems:............................................................................................ 24 LECTURE NO. 03......................................................................................................................... 31 Reading Material................................................................................................................... 31 Overview of Lecture.............................................................................................................. 31 Database Architecture:..................................................................................................... 31 The Architecture:.............................................................................................................. 33 External View (Level, Schema or Model):........................................................................ 35 Conceptual or Logical View:............................................................................................. 37 LECTURE NO. 04......................................................................................................................... 40 Reading Material................................................................................................................... 40 O v er v ie w of L ec t ur e....................................................................................................... 40 In ter n a l or P h ys ic al V i e w / Sc h em a...................................................................... 40 Da ta I n de p en d e nc e :................................................................................................... 43 F u nc t i ons of D B M S.................................................................................................... 45 LECTURE NO. 05......................................................................................................................... 50 Reading Material................................................................................................................... 50 Overview of Lecture.............................................................................................................. 50 Database Development Process...................................................................................... 51 Preliminary Study:............................................................................................................ 51 Database Development Process: Approach 2................................................................. 54 Tools Used for Database System Development:............................................................. 56 Data Flow Diagrams:........................................................................................................ 56 Types of DFD.................................................................................................................... 60 LECTURE NO. 06......................................................................................................................... 63 Reading Material................................................................................................................... 63 Overview of Lecture.............................................................................................................. 63 Detailed Data Flow Diagram:............................................................................................ 63 Data Dictionary................................................................................................................. 64 Database Design Phase................................................................................................... 67 2 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Data Model....................................................................................................................... 68 Types of Data Models....................................................................................................... 68 Types of Database Design............................................................................................... 69 LECTURE NO. 07......................................................................................................................... 70 Reading Material................................................................................................................... 70 Overview of Lecture.............................................................................................................. 70 Entity-Relationship Data Model........................................................................................ 70 The Entity.......................................................................................................................... 71 Classification of entity types............................................................................................. 74 Attribute............................................................................................................................ 75 Types of Attributes............................................................................................................ 77 Summary:......................................................................................................................... 79 Exercises:......................................................................................................................... 79 LECTURE NO. 08......................................................................................................................... 80 Reading Material................................................................................................................... 80 Overview of Lecture.............................................................................................................. 80 Attributes........................................................................................................................... 80 The Keys........................................................................................................................... 80 LECTURE NO. 09......................................................................................................................... 85 Reading Material................................................................................................................... 85 Overview of Lecture.............................................................................................................. 85 Relationships.................................................................................................................... 85 Types of Relationships..................................................................................................... 87 LECTURE NO. 10......................................................................................................................... 91 Reading Material................................................................................................................... 91 Overview of Lecture.............................................................................................................. 91 Roles in Relationships...................................................................................................... 95 Dependencies................................................................................................................... 97 Enhancements in E-R Data Model:.................................................................................. 98 Super-type and Subtypes................................................................................................. 98 Summary:......................................................................................................................... 99 LECTURE NO. 11....................................................................................................................... 100 Reading Material................................................................................................................. 100 Overview of Lecture............................................................................................................ 100 Inheritance Is.................................................................................................................. 100 Super types and Subtypes............................................................................................. 101 Specifying Constraints.................................................................................................... 103 Completeness Constraint............................................................................................... 103 Disjointness Constraint................................................................................................... 104 Subtype Discriminator.................................................................................................... 108 LECTURE NO. 12....................................................................................................................... 110 Reading Material................................................................................................................. 110 Overview of Lecture............................................................................................................ 110 Steps in the Study of system.......................................................................................... 110 LECTURE NO. 13....................................................................................................................... 118 Reading Material................................................................................................................. 118 Overview of Lecture............................................................................................................ 118 3 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Identification of Entity Types of the Examination System.............................................. 118 Relationships and Cardinalities in between Entities....................................................... 120 Conceptual Database Design......................................................................................... 122 Logical Database Design................................................................................................ 122 Conclusion...................................................................................................................... 123 LECTURE NO. 14....................................................................................................................... 124 Reading Material................................................................................................................. 124 Overview of Lecture............................................................................................................ 124 Relational Data Model.................................................................................................... 125 Introduction to the Relational Data model...................................................................... 126 Mathematical Relations.................................................................................................. 129 Database Relations........................................................................................................ 130 Summary........................................................................................................................ 130 Exercise:......................................................................................................................... 131 LECTURE NO. 15....................................................................................................................... 132 Reading Material................................................................................................................. 132 Overview of Lecture............................................................................................................ 132 Database and Math Relations........................................................................................ 132 Degree of a Relation....................................................................................................... 133 LECTURE NO. 16....................................................................................................................... 140 Reading Material................................................................................................................. 140 Overview of Lecture:........................................................................................................... 140 Mapping Relationships................................................................................................... 140 Binary Relationships....................................................................................................... 140 Unary Relationship......................................................................................................... 144 Data Manipulation Languages........................................................................................ 146 Relational Algebra.......................................................................................................... 147 Exercise:......................................................................................................................... 147 LECTURE NO. 17....................................................................................................................... 148 Reading Material................................................................................................................. 148 Overview of Lecture:........................................................................................................... 148 The Project Operator...................................................................................................... 150 LECTURE NO. 18....................................................................................................................... 157 Reading Material................................................................................................................. 157 Overview of Lecture:........................................................................................................... 157 Types of Joins................................................................................................................. 157 Theta Join:...................................................................................................................... 157 Equi–Join:....................................................................................................................... 159 Natural Join:.................................................................................................................... 159 Outer Join:...................................................................................................................... 161 Semi Join:....................................................................................................................... 161 Relational Calculus......................................................................................................... 162 Tuple Oriented Relational Calculus:............................................................................... 162 Domain Oriented Relational Calculus:........................................................................... 162 Normalization.................................................................................................................. 162 LECTURE NO. 19....................................................................................................................... 164 Reading Material................................................................................................................. 164 Overview of Lecture:........................................................................................................... 164 4 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Functional Dependency.................................................................................................. 164 Inference Rules.............................................................................................................. 166 Normal Forms................................................................................................................. 166 Summary........................................................................................................................ 167 Exercise:......................................................................................................................... 168 LECTURE NO. 20....................................................................................................................... 169 Reading Material................................................................................................................. 169 Overview of Lecture:........................................................................................................... 169 Second Normal Form..................................................................................................... 169 Third Normal Form......................................................................................................... 171 Boyce - Codd Normal Form............................................................................................ 173 Higher Normal Forms..................................................................................................... 175 Summary........................................................................................................................ 175 Exercise:......................................................................................................................... 175 LECTURE NO. 21....................................................................................................................... 176 Reading Material................................................................................................................. 176 Overview of Lecture:........................................................................................................... 176 Normalization Summary................................................................................................. 176 Normalization Example................................................................................................... 177 Physical Database Design.............................................................................................. 181 Summary........................................................................................................................ 182 LECTURE NO. 22....................................................................................................................... 183 Overview of Lecture............................................................................................................ 183 The Physical Database Design Considerations and Implementation............................ 183 DESIGNING FIELDS...................................................................................................... 184 CODING AND COMPRESSION TECHNIQUES:........................................................... 185 LECTURE NO. 23....................................................................................................................... 187 Reading Material................................................................................................................. 187 Overview of Lecture............................................................................................................ 187 Physical Record and De-normalization.......................................................................... 187 Partitioning...................................................................................................................... 187 Physical Record and Denormalization........................................................................... 187 Denormalization Situation 1:........................................................................................... 188 Partitioning...................................................................................................................... 189 LECTURE NO. 24....................................................................................................................... 191 Reading Material................................................................................................................. 191 Overview of Lecture............................................................................................................ 191 Vertical Partitioning......................................................................................................... 191 Replication...................................................................................................................... 192 Reduced training cost..................................................................................................... 194 MS SQL Server.............................................................................................................. 194 LECTURE NO. 25....................................................................................................................... 196 Reading Material................................................................................................................. 196 Overview of Lecture............................................................................................................ 196 Rules of SQL Format...................................................................................................... 196 Data Types in SQL Server.............................................................................................. 197 Summary:....................................................................................................................... 200 Exercise:......................................................................................................................... 200 5 © Copyright Virtual University of Pakistan Database Management System (CS403) VU LECTURE NO. 26....................................................................................................................... 201 Reading Material................................................................................................................. 201 Overview of Lecture............................................................................................................ 201 Categories of SQL Commands....................................................................................... 201 Summary........................................................................................................................ 205 Exercise:......................................................................................................................... 205 LECTURE NO. 27....................................................................................................................... 206 Reading Material................................................................................................................. 206 Overview of Lecture............................................................................................................ 206 Alter Table Statement..................................................................................................... 206 LECTURE NO. 28....................................................................................................................... 210 Reading Material................................................................................................................. 210 Select Statement............................................................................................................ 211 Attribute Allias................................................................................................................. 213 LECTURE NO. 29....................................................................................................................... 215 Reading Material................................................................................................................. 215 Overview of Lecture............................................................................................................ 215 Data Manipulation Language.......................................................................................... 215 LECTURE NO. 30....................................................................................................................... 220 Reading Material................................................................................................................. 220 Overview of Lecture............................................................................................................ 220 ORDER BY Clause......................................................................................................... 220 Functions in SQL............................................................................................................ 221 GROUP BY Clause........................................................................................................ 222 HAVING Clause.............................................................................................................. 223 Cartesian Product........................................................................................................... 224 Summary........................................................................................................................ 225 LECTURE NO. 31....................................................................................................................... 226 Reading Material................................................................................................................. 226 Overview of Lecture............................................................................................................ 226 Inner Join........................................................................................................................ 226 Outer Join....................................................................................................................... 228 Semi Join........................................................................................................................ 230 Self Join.......................................................................................................................... 231 Subquery........................................................................................................................ 232 Summary........................................................................................................................ 236 Exercise:......................................................................................................................... 237 LECTURE NO. 32....................................................................................................................... 238 Reading Material................................................................................................................. 238 Overview of Lecture............................................................................................................ 238 Application Programs..................................................................................................... 238 User Interface................................................................................................................. 239 Forms.............................................................................................................................. 240 Tips for User Friendly Interface...................................................................................... 243 LECTURE NO. 33....................................................................................................................... 246 Reading Material................................................................................................................. 246 Overview of Lecture.......................................................................................................... 246 6 © Copyright Virtual University of Pakistan Database Management System (CS403) VU LECTURE NO. 34....................................................................................................................... 255 Reading Material................................................................................................................. 255 Overview of Lecture............................................................................................................ 255 LECTURE NO. 35....................................................................................................................... 260 Reading Material................................................................................................................. 260 Overview of Lecture............................................................................................................ 260 File Organizations........................................................................................................... 260 LECTURE NO. 36....................................................................................................................... 265 Reading Material................................................................................................................. 265 Overview of Lecture............................................................................................................ 265 Hashing........................................................................................................................... 265 Hash Functions............................................................................................................... 266 Hashed Access Characteristics...................................................................................... 266 Mapping functions.......................................................................................................... 266 Open addressing:........................................................................................................... 269 LECTURE NO. 37....................................................................................................................... 270 Reading Material................................................................................................................. 270 Overview of Lecture:........................................................................................................... 270 Index............................................................................................................................... 270 Index Classification......................................................................................................... 272 Summary........................................................................................................................ 274 LECTURE NO. 38....................................................................................................................... 275 Reading Material................................................................................................................. 275 Overview of Lecture............................................................................................................ 275 Ordered Indices.............................................................................................................. 275 Clustered Indexes........................................................................................................... 275 Non-clustered Indexes.................................................................................................... 276 Dense and Sparse Indices............................................................................................. 276 Multi-Level Indices.......................................................................................................... 277 LECTURE NO. 39 AND 40......................................................................................................... 280 Reading Material................................................................................................................. 280 Overview of Lecture............................................................................................................ 280 Views.............................................................................................................................. 280 To Focus on Specific Data............................................................................................. 280 Characteristics /Types of Views:.................................................................................... 283 Characteristics of Views................................................................................................. 286 LECTURE NO. 41....................................................................................................................... 288 Reading Material................................................................................................................. 288 Overview of Lecture............................................................................................................ 288 Updating Multiple Tables................................................................................................ 288 Materialized Views.......................................................................................................... 289 Transaction Management............................................................................................... 291 LECTURE NO. 42....................................................................................................................... 293 Reading Material................................................................................................................. 293 Overview of Lecture............................................................................................................ 293 The Concept of a Transaction........................................................................................ 293 Transactions and Schedules.......................................................................................... 294 7 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Concurrent Execution of Transactions........................................................................... 295 Serializability................................................................................................................... 296 Lock-Based Concurrency Control................................................................................... 297 Deadlocks....................................................................................................................... 299 LECTURE NO. 43....................................................................................................................... 302 Reading Material................................................................................................................. 302 Overview of Lecture............................................................................................................ 302 Incremental Log with Deferred Updates......................................................................... 302 Incremental Log with Immediate Updates...................................................................... 305 Concurrency Control....................................................................................................... 307 Summary........................................................................................................................ 308 LECTURE NO. 44....................................................................................................................... 310 Reading Material................................................................................................................. 310 Overview of Lecture............................................................................................................ 310 Uncommitted Update Problem....................................................................................... 310 Inconsistent Analysis...................................................................................................... 311 Serial Execution.............................................................................................................. 312 Serializability................................................................................................................... 315 Locking........................................................................................................................... 315 Summary........................................................................................................................ 316 LECTURE NO. 45....................................................................................................................... 318 Reading Material................................................................................................................. 318 Overview of Lecture:........................................................................................................... 318 Locking Idea................................................................................................................... 319 DeadLock........................................................................................................................ 320 DeadLock Handling........................................................................................................ 320 Wait – for Graph:............................................................................................................ 320 Deadlock Resolution....................................................................................................... 323 Timestamping rules........................................................................................................ 324 8 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Lecture No. 01 Reading Material “Database Systems Principles, Design and Implementation” Chapter 1. written by Catherine Ricardo, Maxwell Macmillan. Overview of Lecture o Introduction to the course o Database definitions o Importance of databases o Introduction to File Processing Systems o Advantages of the Database Approach Introduction to the course This course is first (fundamental) course on database management systems. The course discusses different topics of the databases. We will be covering both the theoretical and practical aspects of databases. As a student to have a better understanding of the subject, it is very necessary that you concentrate on the concepts discussed in the course. Areas to be covered in this Course: o Database design and application development: How do we represent a real-world system in the form of a database? This is one major topic covered in this course. It comprises of different stages, we will discuss all these stages one by one. o Concurrency and robustness: How does a DBMS allow many users to access data concurrently, and how does it protect against failures? o Efficiency and Scalability: How does the database cope with large amounts of data? 9 © Copyright Virtual University of Pakistan Database Management System (CS403) VU o Study of tools to manipulate databases: In order to practically implement, that is, to perform different operations on databases some tools are required. The operations on databases include right from creating them to add, remove and modify data in the database and to access by different ways. The tools that we will be studying are a manipulation language (SQL) and a DBMS (SQL Server). Database definitions: Definitions are important, especially in technical subjects because definition describes very comprehensively the purpose and the core idea behind the thing. Databases have been defined differently in literature. We are discussing different definitions here, if we concentrate on these definitions, we find that they support each other and as a result of the understanding of these definitions, we establish a better understanding of use, working and to some extent the components of a database. Def 1: A shared collection of logically related data, designed to meet the information needs of multiple users in an organization. The term database is often erroneously referred to as a synonym for a “database management system (DBMS)”. They are not equivalent and it will be explained in the next section. Def 2: A collection of data: part numbers, product codes, customer information, etc. It usually refers to data organized and stored on a computer that can be searched and retrieved by a computer program. Def 3: A data structure that stores metadata, i.e. data about data. More generally we can say an organized collection of information. Def 4: A collection of information organized and presented to serve a specific purpose. (A telephone book is a common database.) A computerized database is an updated, organized file of machine readable information that is rapidly searched and retrieved by computer. Def 5: An organized collection of information in computerized format. Def 6: A collection of related information about a subject organized in a useful manner that provides a base or foundation for procedures such as retrieving information, drawing conclusions, and making decisions. Def 7: A Computerized representation of any organizations flow of information and storage of data. Each of the above given definition is correct, and describe database from slightly variant perspectives. From exam point of view, anyone will do. However, within this course, we will be referring first of the above definitions more frequently, and concepts discussed in the definition like, logically related data, shared collection should be clear. Another 10 © Copyright Virtual University of Pakistan Database Management System (CS403) VU important thing that you should be very clear about is the difference between database and the database management system (DBMS). See, the database is the collection of data about anything, could be anything. Like cricket teams, students, busses, movies, personalities, stars, seas, buildings, furniture, lab equipment, hobbies, hotels, pets, countries, and many more anything about which you want to store data. What we mean by data; simply the facts or figures. Following table shows the things and the data that we may want to store about them: Thing Data (Facts or figures) Cricket Player Country, name, date of birth, specialty, matches played, runs etc. Scholars Name, data of birth, age, country, field, books published etc. Movies Name, director, language (Punjabi is default in case of Pakistan) etc. Food Name, ingredients, taste, preferred time, origin, etc. Vehicle Registration number, make, owner, type, price, etc. There could be infinite examples, and please note that the data that is listed about different things in the above table is not the only data that can be defined or stored about these things. As has been explained in the definition one above, there could be so many facts about each thing that we are storing data about; what exactly we will store depends on the perspective of the person or organization who wants to store the data. For example, if you consider food, data required to be stored about the food from the perspective of a cook is different from that of a person eating it. Think of a food, like, Karhahi Ghost, the facts about Karhahi ghosht that a cook will like to store may be, quantity of salt, green and red chilies, garlic, water, time required to cook and like that. Where as the customer is interested in chicken or meat, then black or red chilies, then weight, then price and like that. Well, definitely there are some things common but some are different as well. The thing is that the perspective or point of view creates the difference in what we store; however, the main thing is that the database stores the data. The database management system (DBMS), on the other hand is the software or tool that is used to manage the database and its users. A DBMS consist of different components or subsystem that we will study about later. Each subsystem or component of the DBMS performs different function(s), so a DBMS is collection of different programs but they all work jointly to manage the data stored in the database and its users. In many books and may be in this course sometimes database and database management system are used interchangeably but there is a clear difference and we should be clear about them. Sometimes another term is used, that is, the database system, again, this term has been used differently by different people, however in this course we use the term database system as a combination of database and the database management system. So database is collection of data, DBMS is tool to manage this data, and both jointly are called database system. 11 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Importance of the Databases Databases are important; why? Traditionally computer applications are divided into commercial and scientific (or engineering) ones. Scientific applications involve more computations, that is, different type of calculations that vary from simple to very complex. Today such applications exist, like in the fields of space, nuclear, medicine that take hours or days of computations on even computers of the modern age. On the other hand, the applications that are termed as commercial or business applications do not involve much computations, rather minor computation but mainly they perform the input/output operations. That is, these applications mainly store the data in the computer storage, then access and present it to the users in different formats (also termed as data processing) for example, banks, shopping, production, utilities billing, customer services and many others. As is clear from the example systems mentioned, the commercial applications exist in the day to day life and are related directly with the lives of common people. In order to manage the commercial applications more efficiently databases are the ultimate choice because efficient management of data is the sole objective of the databases. So such applications are being managed by databases even in a developing country like Pakistan, yet to talk about the developed countries. This way databases are related directly or indirectly almost every person in society. Databases are not only being used in the commercial applications rather today many of the scientific/engineering application are also using databases less or more. databases are concern of the effectively latter form of applications are more Commercial applications involve The goal of this course is to present an in-depth introduction to databases, with an emphasis on how to organize information in the database and to maintain it and retrieve it efficiently, that is, how to design a database and use it effectively. Databases and Traditional File Processing Systems Traditional file processing system or simple file processing system refers to the first computer-based approach of handling the commercial or business applications. That is why it is also called a replacement of the manual file system. Before the use computers, the data in the offices or business was maintained in the files (well in that perspective some offices may still be considered in the pre-computer age). Obviously, it was laborious, time consuming, inefficient, especially in case of large organizations. Computers, initially designed for the engineering purposes were though of as blessing, since they helped efficient management but file processing environment simply transformed manual file work to computers. So processing became very fast and efficient, but as file processing systems were used, their problems were also realized and some of them were very severe as discussed later. It is not necessary that we understand the working of the file processing environment for the understanding of the database and its working. However, a comparison between the characteristics of the two definitely helps to understand the advantages of the databases 12 © Copyright Virtual University of Pakistan Database Management System (CS403) VU and their working approach. That is why the characteristics of the traditional file processing system environment have been discussed briefly here. Fig. 1: A typical file processing environment The diagram presents a typical traditional file processing environment. The main point being highlighted is the program and data interdependence, that is, program and data depend on each other, well they depend too much on each other. As a result any change in one affects the other as well. This is something that makes a change very painful or problematic for the designers or developers of the system. What do we mean by change and why do we need to change the system at all. These things are explained in the following. The systems (even the file processing systems) are created after a very detailed analysis of the requirements of the organizations. But it is not possible to develop a system that does not need a change afterwards. There could be many reasons, mainly being that the users get the real taste of the system when it is established. That is, users tell the analysts or designers their requirements, the designers design and later develop the system based on those requirements, but when system is developed and presented to the users, it is only then they realize the outcome of the effort. Now it could be slightly and (unfortunately) sometimes very different from what they expected or wanted it to be. So the users ask changes, minor or major. Another reason for the change is the change in the requirements. For example, previously the billing was performed in an organization on the monthly 13 © Copyright Virtual University of Pakistan Database Management System (CS403) VU basis, now company has decided to bill the customers after every ten days. Since the bills are being generated from the computer (using file processing system), this change has to be incorporated in the system. Yet another example is that, initially bills did not contain the address of the customer, now the company wants the address to be placed on the bill, so here is change. There could be many more examples, and it is so common that we can say that almost all systems need changes, so system development is always an on-going process. So we need changes in the system, but due to program-data interdependence these changes in the systems were very hard to make. A change in one will affect the other whether related or not. For example, suppose data about the customer bills is stored in the file, and different programs use this file for different purposes, like adding data into the bills file, to compute the bill and to print the bill. Now the company asks to add the customers’ address in the bills, for this we have to change the structure of the bill file and also the program that prints the bill. Well, this was necessary, but the painful thing is that the other programs that are using these bills files but are not concerned with the printing of the bills or the change in the bill will also have to be changed, well; this is needless and causes extra, unnecessary effort. Another major drawback in the traditional file system environment is the non-sharing of data. It means if different systems of an organization are using some common data then rather than storing it once and sharing it, each system stores data in separate files. This creates the problem of redundancy or wastage of storage and on the other hand the problem on inconsistency. The change in the data in one system sometimes is not reflected in the same data stored in other system. So different systems in organization; store different facts about same thing. This is inconsistency as is shown in figure below. Fig. 2: Some more problems in File System Environment 14 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Previous section highlighted the file processing system environment and major problems found there. The following section presents the benefits of the database systems. Advantages of Databases It will be helpful to reiterate our database definition here, that is, database is a shared collection of logically related data, designed to meet the information needs of multiple users in an organization. A typical database system environment is shown in the figure 3 below: Fig. 3: A typical Database System environment The figure shows different subsystem or applications in an educational institution, like library system, examination system, and registration system. There are separate, different application programs for every application or subsystem. However, the data for all applications is stored at the same place in the database and all application programs, relevant data and users are being managed by the DBMS. This is a typical database system environment and it introduces the following advantages: o Data Sharing The data for different applications or subsystems is placed at the same place. This introduces the major benefit of data sharing. That is, data that is common among different applications need not to be stored repeatedly, as was the case in the file processing environment. For example, all three systems of an educational institution shown in figure 3 need to store the data about students. The example data can be seen 15 © Copyright Virtual University of Pakistan Database Management System (CS403) VU from figure 2. Now the data like registration number, name, address, father name that is common among different applications is being stored repeatedly in the file processing system environment, where as it is being stored just once in database system environment and is being shared by all applications. The interesting thing is that the individual applications do not know that the data is being shared and they do not need to. Each application gets the impression as if the data is being for stored for it. This brings the advantage of saving the storage along with others discussed later. o Data Independence Data and programs are independent of each other, so change is once has no or minimum effect on other. Data and its structure is stored in the database where as application programs manipulating this data are stored separately, the change in one does not unnecessarily effect other. o Controlled Redundancy Means that we do not need to duplicate data unnecessarily; we do duplicate data in the databases, however, this duplication is deliberate and controlled. o Better Data Integrity Very important feature; means the validity of the data being entered in the database. Since the data is being placed at a central place and being managed by the DBMS, so it provides a very conducive to check or ensure that the data being entered into the database is actually valid. Integrity of data is very important, since all the processing and the information produced in return are based on the data. Now if the data entered is not valid, how can we be sure that the processing in the database is correct and the results or the information produced is valid? The businesses make decisions on the basis of information produced from the database and the wrong information leads to wrong decisions, and business collapse. In the database system environment, DBMS provides many features to ensure the data integrity, hence provides more reliable data processing environment. Dear students, that is all for this lecture. Today we got the introduction of the course, importance of the databases. Then we saw different definitions of database and studied what is data processing then studied different features of the traditional file processing environment and database (DB) system environment. At the end of lecture we were discussing the advantages of the DB approach. There some others to be studied in the next lecture. Suggestions are welcome. Exercises o Think about the data that you may want to store about different things around you o List the changes that may arise during the working of any system, lets say Railway Reservation System 16 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Lecture No. 02 Reading Material “Database Systems Principles, Design and Implementation” written by Catherine Ricardo, Maxwell Macmillan. Overview of Lecture o Some Additional Advantages of Database Systems o Costs involved in Database systems o Levels of data o Database users Difference between Data and Information Data is the collection of raw facts collected from any specific environment for a specific purpose. Data in itself does not show anything about its environment, so to get desired types of results from the data we transform it into information by applying certain processing on it. Once we have processed data using different methods data is converted into meaningful form and that form of the Data is called information Example: Example: Fig. 1: Data and Information 17 © Copyright Virtual University of Pakistan Database Management System (CS403) VU If we consider the data in the above figure without the titles or the labels associated with the data (EmpName, age, salary) then it is not much useful. However, after attaching these labels it brings some meanings to us, this meaningfulness is further increased when we associate some other labels, like the company name and the department name etc. So this is a very simple example of processing that we can do on the data to make it information. Once we have clear idea of what data and information is we proceed with another term knows as “schema” Schema is a repository or structure to express the format and other different information about data and database, as we can see from the database definition “Database is a self describing collection of interrelated records.” The word self describing means that the data storage and retrieval mechanism and its format is described in the database, Actual place where these definitions and descriptions are performed is database schema. o Database Application: Database Application is a program or group of programs which is used for performing certain operations on the data stored in the database. These operations may contain insertion of data into a database or extracting some data from the database based on a certain condition, updating data in the database, producing the data as output on any device such as Screen, disk or printer. o Database Management Systems: Database management system is software of collection of small programs to perform certain operation on data and manage the data. Two basic operations performed by the DBMS are: Management of Data in the Database Management of Users associated with the database. Management of the data means to specify that how data will be stored, structured and accessed in the database. Management of database users means to manage the users in such a way that they can perform any desired operations on the database. DBMS also ensures that a user can not perform any operation for which he is not allowed. And also an authorized user is not allowed to perform any action which is restricted to that user. In General DBMS is a collection of Programs performing all necessary actions associated to a database. 18 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Further Advantages of Database Systems: Database systems are very much beneficent to enterprises and businesses, some of the advantages are listed below: o Data consistency o Better data security o Faster development of new applications o Economy of scale o Better concurrency control o Better backup and recovery procedures o Data Consistency: Data consistency means that the changes made to different occurrence of data should be controlled and managed in such a way that all the occurrences have same value for any specific data item. Data inconsistency leads to a number of problems, including loss of information and incorrect results. In database approach it is controlled because data is shared and consistency is controlled and maintained. o Better Data Security: All application programs access data through DBMS, So DBMS can very efficiently check that which user is performing which action and accessing which part of data , So A DBMS is the most effectively control and maintain security of Data stored in a database. o Faster Application Development: The database environment allows us faster application development because of its many reasons. As we know that database is designed with the factor of future development in mind So whenever we have to build a new application to meet the growing needs of the computerized environment, it may be easy due to the following reason: The data needed for the new application already resides in the database. 19 © Copyright Virtual University of Pakistan Database Management System (CS403) VU The data might not already reside in the database but it could be derived from the data present in the database Thus we can say that, to develop a new application for an existing database system less effort is required in terms of the system and database design. o Economy of Scale: Databases and database systems are designed to share data stored in one location for many different purposes, So it needs not be stored as many number of times in different forms as it is used, for example the data used by Admission Department of any education institution can be used to maintain the attendance record of the students as well as the examination records of the students. So it saves us lots of efforts and finances providing economy of scale. o Better Concurrency Control: Concurrency means the access of database form as number of points simultaneously. Concurrency control means to access the database in such a way that all the data accesses are completed correctly and transparently. One example of controlled concurrency is the use of ATM Machine for withdrawal of money (cash). All ATM machines of a bank are interconnected to a central database system worldwide, so that a user can access its account from anywhere in the world and can get cash from any ATM terminal. As there are thousands of ATM terminal across the world for a specific bank so as a result thousands of user process and access the bank’s database. All this process is managed concurrently using the database systems and is done in such an efficient manner that no two user face any delay in the processing of their requests. o Better Backup and Recovery Facility: Data is a very important resource and is very much valuable for any organization, loss of such a valuable resource can result in a huge strategic disasters. As Data is stored on today’s’ storage devices like hard disks etc., It is necessary to take periodic backups of data so that in case a storage device looses the data due to any damage we should be able 20 © Copyright Virtual University of Pakistan Database Management System (CS403) VU to restore the data a nearest point, Database systems offer excellent facilities for taking backup of data and good mechanism of restoring those backups to get back the backed-up data. It some time happens that a database which was in use and very important transactions were made after the last backup was made, all of a sudden due to any disastrous situation the database crashes (improper shutdown, invalid disk access, etc.) Now in such a situation the database management system should be able to recover the database to a consistent state so that the transactions made after the last backup are not lost. Cost Involved: Enjoying all these benefits of the database systems do have some additional costs on any organization which is going to adopt a database environment. These charges may also be known as the disadvantages of the database system. Different types of costs (Financial and Personnel) which an organization faces in adopting a database system are listed below: o High Cost: Database Systems have a number of inherent charges which are to be born by any organization that is going to adopt it. High Cost is one of these inherent charges, it includes the need for specialized software which is used to run database systems, Additional and specialized hardware and technically qualified staff are the requirements for adopting to the database system, all these requirements need an organization to invest handsome amount of money to have all the requirements of the database systems. o Conversion Cost: Once an organization has decided to adopt database system for its operations, it is not only the finance and technical man-power which is required for switching on to database system, it further has some conversion charges needed for adopting the database system, this is also a very important stage for making decision about the way the system will be converted to database system. 21 © Copyright Virtual University of Pakistan Database Management System (CS403) VU o Difficult Recovery Procedures: Although the database systems and database management systems provide very efficient ways of data recovery in case of any disaster, still the process of recovering a crashed database is very much technical and needs good professional skills to perform a perfect recovery of the database. Importance of Data o Data as a Resource: A resource is anything which is valuable for an organization. There can be a number of resources in any organization, for example, Buildings, Furniture, Vehicle, Technical Staff, Managers, supporting staff and Machinery etc. As all these are resources for organizations and are consumed very much carefully to get full benefit out of them, Data in the same way is a very important resources and needs to considered equally important as other resource are considered. Why we call data as a resource? Data is truly considered a resource because for an organization to make proper decisions at proper time it is only the data which can provide correct information and in-turn cause good utilization of other organizational resources. Organizations can not make good and effective decisions if the required data is not available in time or in the correct and desired format, such bad and miscalculated decisions ultimately lead to the failure of organizations or business. Levels of Data o Real World Data The real world level of data means that level of data at which entities or objects exist in reality, it means that any object existing in reality have a name and other identifiable attributes through which we can identify that specific object or entity. Example: Any Student 22 © Copyright Virtual University of Pakistan Database Management System (CS403) VU o Meta Data: For storage of the data related to any entity or object existing at real world level we define the way the data will be stored in the database. This is called Meta data. Meta data is also known as schema for the real world data. It tells that what type of data will be stored in the database what will be size of a certain attribute of the real world data, how many and what attributes will be used to store the data about the entity in the database. Example: Name , Character Type, 25 character size field, Age, Date type, 8 bytes size Class, Alpha Numeric, 8 byte size field o Existence of Data: Existence of the data level shows the actual data regarding the entities as real world level according to the rules define at the Meta Data level. Example: According to the definition given in the Meta data level the Actual data or Data occurance for the entity at real world level is shown below: Name Age Class Ali 20/8/1979 MCS-I Amir 22/3/1978 MCS-II etc… 23 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Fig. 2: Levels of Data Users of Database Systems: o Application Programmers o End Users Naïve Sophisticated o Application programmers: This category of database users contains those people who create different types of database application programs that we have seen earlier. Application programmers design the application according to the needs of the other users of the database in a certain environment. Application programmers are skilled people who have clear idea of the structure of the database and know clearly about the needs of the organizations. o End Users: 24 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Second category of the Database users are the end users, this group of users contains the people who use the database application programs developed by the Application programmers. This category further contains two types of users Naïve Users Sophisticated Users Naïve Users This category of users is that category who simply use the application database programs created by the programmers. This groups has no interaction with other parts of there database and only use the programs meant for them. They have not to worry about the further working of the database. Sophisticated Users: This type of users has some additional rights over the Naïve users, which means that they can access the data stored in the database any of their desired way. They can access data using the application programs as well as other ways of accessing data. Although this type of users has more rights to access data, but these users have to take more responsibility and they need to be aware of the database structure. Moreover such users should be skilled enough to be able to get data from database with making and damage or loss to the data in database. o Database Administrators (DBA): This class of database users is the most technical class of db users. They need to have the knowledge of how to design and manage the database use as well as to manage the data in the database. DBA is a very responsible position in an organization. He is responsible for proper working of the database and DBMS, has the responsibility of making proper database backups and make necessary actions for recovering the database in case of a database crash. To fulfill the requirements of a DBA position a DBA needs vast experience and very elegant technical skills. 25 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Duties of the DBA A Database administrator has some very precisely defined duties which need be performed by the DBA very religiously. A short account of these jobs is listed below: o Schema definition o Granting data access o Routine Maintenance Backups Monitoring disk space Monitoring jobs running o Schema Design DBA in some organization is responsible for designing the database schema, which means that DBA is the person who create all the meta Data information for the organization on which the database is based. However in some very large scale organizations this job is performed by the Database designer, which is hired for the purpose of database Design and once the database system, is installed and working it is handed over to the DBA for further operation. o Granting Access to Users: DBA is also responsible for grant of access rights to the database users. Along with granting and revoking (taking back) the rights the DBA continuously monitors and ensure the legal use of these rights. o Monitoring Disk Space : When a new database is created it takes a limited space but as a result of daily activity the database acquires more data and grows in size very rapidly. The DBA has to monitor the disk space usage and statistics to ensure that no data over flow occurs at any stage. o Monitoring Running Jobs: To ensure the secure and proper functioning of the database system a DBA continuously monitors some associated activities also and ensure that all users are using their 26 © Copyright Virtual University of Pakistan Database Management System (CS403) VU authorities legally and different devices attached to the database system are functioning properly. Typical Components of a Database Environment: Different typical components of a database environment are shown in the figures below; they describe graphically the role of different types of users. Fig. 3: DBMS and Database Database is used to store data and DBMS uses mechanisms to get data from the database 27 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Fig. 4: Application Programs Application programs talk to DBMS and ask for the data required Fig. 5: Database Designers Database designers design (for large organizations) the database and install the DBMS for use by the users of the database in any specific organization. 28 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Fig. 6: Database Administrator Once Database has been installed and is functioning properly in a production environment of an organization the Database Administrator takes over the charge and performs specific DBA related activities including: o Database maintenance. o Database Backup. o Grant of rights to database users. o Monitoring of Running Jobs o Managing Print jobs o Ensuring quality of Service to all users. 29 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Fig. 7: Database Administration’s interaction with other users o Database administrator can interact with the database designer during database design phase so that he has a clear idea of the database structure for easy reference in future. o This helps DBA perform different tasks related to the database structure. o DBA also interacts with the application programmers during the application development process and provides his services for better design of applications. o End users also interact with the system using application programs and other tools as specified in the description above. This concludes lecture number 2, in case of any queries, please feel free to contact. 30 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Lecture No. 03 Reading Material “Database Systems Principles, Design and Implementation” Section: 4.1.1, written by Catherine Ricardo, Maxwell Macmillan. 4.1.2 Overview of Lecture o Database Architecture o External View of the database o Conceptual view of the database Database Architecture: Standardization of database systems is a very beneficent in terms of future growth, because once a system is defined to follow a specific standard, or is built on a specific standard, it provides us the ease of use in a number of aspects. First if any organization is going to create a new system of the same usage shall create the system according to the standards and it will be easier to develop, because the standards which are already define will be used for developing the system. Secondly if any organization wants to create and application software that will provide additional support to the system, it will be an easier task for them to develop such system and integrate them into existing database applications. Users which will be using the system will be comfortable with the system because a system built on predefined standards is easy to understand and use, rather than understanding learning and using an altogether new system which is designed and built without following any standards. 31 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Expansion to systems which are not built on standards is very hard and needs lots of efforts. Technical staff working on a system built on standard has no problem to learn the use and architecture of the system and whenever there is a need in change of staff new staff members can be hired and put to work without any prior training for the use of system. Database standard proposed by ANSI SPARK in 1975 is being used worldwide and is the only most popular agreed upon standard for database systems. The Three Level Schema architecture provides us a number of benefits. For accessing data at different levels we have a number of users because not all users have to access data in database at all the database levels. The 3 levels architecture allows us to separate the physical representation of data from the users’ views of data. In the database, same data is stored in a specific feasible format and is available to different users in different formats as desired by different users. For example, consider we have stored the DOB (Date of Birth) in the database in a particular format, like in the form of dd-mm-yyyy (for example, 28-03-1987). However, the users from different departments may require to view the date of birth in different forms; the examination department may ask it to be displayed as month-day-yyyy (like march-28-1987) the Registrar’s office may ask to display date of birth as mm/dd/yyyy, still the Library may need the in the form of dd/mm/yy. The Three Level Schema allows us to access the data in different formats at the external level, which is stored in a specific format at the internal level. The Three levels architecture is useful for hiding the details of internal systems; it in-fact hides the details of underlying system views from the users at other levels and restricts the access of data and the system from any unauthorized intervention. It is the mechanism which allows us to store the data in the system in such a way that it can be provided to all users in their desired formats and with unveiling other details and information stored in the database. Moreover if there is a change to be done to the data stored in the database subject to the 32 © Copyright Virtual University of Pakistan Database Management System (CS403) VU requirements of a specific user it needs not be changed for that user specifically, we can create a change to the specific external view of that user and the internal details remain unchanged. Also if we want to change the underlying storage mechanism of the data stored on the disk we can do it without affecting the internal and conceptual view at the lowest level in the three levels architecture is the internal view or internal level which is shown below in the diagram and is illustrated in the coming lines. Fig. 1: Three level architecture of database The Architecture: The schemas as it has been defined already; is the repository used for storing definitions of the structures used in database, it can be anything from any entity to the whole organization. For this purpose the architecture defines different schemas stored at different levels for isolating the details one level from the other. 33 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Different levels existing at different levels of the database architecture are expressed below with emphasis on the details of all the levels individually. Core of the database architecture is the internal level of schema which is discussed a bit before getting into the details of each level individually. The internal level implements all the inner details and defines the intentions of the database. Internal schema or view defines the structures of the data and other data related activities in the database. For example it defines that for a student what data will be stored in terms of attributes of the student and it also defines how different values for these attributes will be stored, also it tells that who is allowed to make changes to the database and what changes he can make, etc. These details give us the internal schema and are called the intention of the database. Intention for a database is almost permanent, because while designing the database it is ensured that no information is left behind which is important enough to be stored in the database and what information is important to be stored in the database from the future point of view. Once the intention of the database has been defined then it is undesirable to change the intention for any reason. Because any small change in the intention of the database may need a lot of changes to be made to the data stored in the database. Extension of the database is performed on the bases of a complete intention, i-e once a database has been defined it is populated with the data of the organization for which the database is created. This population of the database is also called as the extension of the database. Extension is always done according to the rules defined in the internal schema design or the intention of the database. Effects of changes made to different levels of the database architecture: We can make changes to the different levels of the database but these changes need very serious consideration before they are made, Changes at different levels of database architecture need different levels of users attention for example a change to the data made for the extension of data will effect only a single record whereas when we make a change to the internal level of the 34 © Copyright Virtual University of Pakistan Database Management System (CS403) VU database the change effects all the stored records, similarly an invalid change in the extension of the database is not that fatal as a change in the intention of the database because a change in the extension of the database is not very hard to undo; incase of a mishap whereas a change of the same magnitude to the intention of the database might cause a large number of database errors (inconsistencies and data loss). External View (Level, Schema or Model): This level is explicitly an end user level and presents data as desired by the users of the database. As it is known that the database users are classified on two grounds o Section of the organization o Nature of Job of the users The external level of the database caters to the needs of all the database users starting from a user who can view the data only which is of his concern up-to the users who can see all the data in the database and make all type of actions on that data. External level of the database might contain a large number of user views, each user view providing the desired features and fulfilling requirements for the user or user group for which it is intended. The restriction or liberty a user or user groups get in his rights is the external view of that user groups and is decided very carefully. External views are also helpful when we want to display the data which is not place in the database or not stored at all. Example of the first case can be a customer Phone number stored in the database. But when contacting the person it might appear that the area code for that specific user is not stored in the database, in that case we can simply pick up the area or city id of the customer and find the area code for that city from the corresponding Area Codes table. Another situation may arise when we want to get a student enrolled in an institution and want to make sure that the student qualifies for the minimum required age limit, we will look the database, for the students age but if we have 35 © Copyright Virtual University of Pakistan Database Management System (CS403) VU stored only the date of birth of the student then the age of the student needs to be calculated at that very instance; this can be done very easily in the specific user view and age of the student can be calculated, even the user-view itself can tell use whether the student qualifies for the admission or not. As the user view is the only entity or the interface through which a user will operate the database or use it so it must be designed in such a way that it is easy to use and easy to manage and self descriptive, also it is easy to navigate through. Also it should not allow the user to get or retrieve data which is not allowed to the user, so the user view should both be a facilitator and also a barrier for proper utilization of the database system. As the system grows it is possible that a user view may change in structure, design and the access it provides to the users. SO External views are designed and create in way that they can be modified at a later stage without making any changes in the logical or internal views. In the diagram below we can see two different users working as end users having their own external view; we can see that the same data record is displayed in two entirely different ways. Fig. 2: Mapping between External layer and lower layers 36 © Copyright Virtual University of Pakistan Database Management System (CS403) VU Conceptual or Logical View: This is the level of database architecture which contains the definition of all the data to be stored in the database and also contains rules and information about that structure and type of that data. The conceptual view is the complete description of the data stored in the database. It stores the complete data of the organization that is why it is also known as the community view of the database. The conceptual view shows all the entities existing in the organization, attribute or characteristics associated with those entities and the relationships which exist among the entities of the organization. We can take the example of the customers of a company. Now the conceptual schema will have all the details of the products of the company, retailing stores of the company, products present in the stock, products which are ready to be delivered, salespersons of the company, manager of the company and literally every other thing which is associated with the business of the company in any way. Now after having all the information we know that the customers buy products from the outlets of the company, thus in such a case a specific customer has a relationship with that specific outlet of the company, or the customer may be represented as having association with the sales person which in-turn has association with the outlet., there may be a number of customers at a certain outlet and also to mange these salespersons there will be one or more managers. We can see from the above given scenario that all the entities are logically related to each other in way or the other. The conceptual schema actually manages all such relationship and maps these relationships among the member entities. Conceptual schema along-with having all the information which is to be stored in the database stores the definition of the data to be stored. The definition may contain types of data, and constraints on data values etc. 37 © Copyright Virtual University of Pakistan Database Management System (CS403)