Introduction To Databases Lecture 1 PDF

CS2031 INTRODUCTION TO DATABASES Dr. Thamir M. Qadah [email protected] Course Website We w ill u se Bla ckb o a rd a n d o u r co u rse w eb site. CHAPTER 1 Databases and Database Users Slide 1- 3 OUTLINE Ty p e s o f D a t a b a s e s a n d D a t a b a s e A p p l i c a t i o...

CS2031 INTRODUCTION TO DATABASES Dr. Thamir M. Qadah [email protected] Course Website We w ill u se Bla ckb o a rd a n d o u r co u rse w eb site. CHAPTER 1 Databases and Database Users Slide 1- 3 OUTLINE Ty p e s o f D a t a b a s e s a n d D a t a b a s e A p p l i c a t i o n s Basic Definitions Ty p i c a l D B M S F u n c t i o n a l i t y Example of a Database (UNIVERSITY) Main Characteristics of the Database Approach Ty p e s o f D a t a b a s e U s e r s A d va n t a g e s o f U s i n g t h e D a t a b a s e A p p r o a c h H i s t o r i c a l D e v e l o p m e n t o f D a t a b a s e Te c h n o l o g y Extending Database Capabilities When Not to Use Databases Slide 1- 4 Types of Databases and Database Applications Traditional Applications: Nu me r i c a n d T e x t u a l D a t a b a s e s Mo r e R e c e n t Ap p l i c a t i o n s : Mu l t i me d i a D a t a b a s e s G e o g r a p h i c I n f o r ma t i o n S y s t e ms ( G I S ) B i o l o g i c a l a n d G e n o me D a t a b a s e s D a t a Wa r e h o u s e s Mo b i l e d a t a b a s e s R e a l - t i me a n d A c t i v e D a t a b a s e s First part of book focuses on traditional applications A n u mb e r o f r e c e n t a p p l i c a t i o n s a r e d e s c r i b e d l a t e r i n t h e b o o k ( f o r e x a mp l e , C h a p t e r s 2 4 , 2 5 , 2 6 , 2 7 , 2 8 , 2 9 ) Slide 1- 5 Recent Developments (1) Social Networks started capturing a lot of information about people and about communications among people-posts, tweets, photos, videos in systems such as: - Facebook - Twitter - Linked-In All of the above constitutes data Search Engines- Google, Bing, Yahoo : collect their own repository of web pages for searching purposes Slide 1- 6 Recent Developments (2) NEW TECHNOLOGIES BIG DATA STORAGE NOSQL (NOT ONLY A LARGE AMOUNT OF ARE EMERGING FROM SYSTEMS INVOLVING SQL) SYSTEMS DATA NOW RESIDES THE SO-CALLED NON- LARGE CLUSTERS OF (CHAPTER 24) ON THE “CLOUD” DATABASE SOFTWARE DISTRIBUTED WHICH MEANS IT IS IN VENDORS TO MANAGE COMPUTERS (CHAPTER HUGE DATA CENTERS VAST AMOUNTS OF 25) USING THOUSANDS OF DATA GENERATED ON MACHINES. THE WEB: Slide 1- 7 Basic Definitions Da t a b a s e : A collection of related data. Da t a : K n o w n f a c t s t h a t c a n b e r e c o r d e d a n d h a v e a n i mp l i c i t me a n i n g. Mi n i - wo r l d : S o me p a r t o f t h e r e a l w o r l d a b o u t w h i c h d a t a i s s t o r e d i n a d a t a b a s e. F o r e x a mp l e , s t u d e n t g r a d e s a n d t r a n s c r i p t s a t a university. Da t a b a s e Ma n a g e me n t S y s t e m ( DB MS ) : A s o f t w a r e p a c k a g e / s y s t e m t o f a c i l i t a t e t h e c r e a t i o n a n d ma i n t e n a n c e o f a c o mp u t e r i z e d d a t a b a s e. Da t a b a s e S y s t e m: T h e D B MS s o f t w a r e t o g e t h e r w i t h t h e d a t a i t s e l f. S o me t i me s , t h e a p p l i c a t i o n s a r e a l s o i n c l u d e d. Slide 1- 8 Impact of Databases and Database Technology B u s i n e s s e s : B a n k i n g , In s u ra n c e , Re t a i l , Tra n s p o r t a t i o n , H e a l t h c a r e , Manufacturing S e r v i c e In d u s t r i e s : F i n a n c i a l , Re a l - e s t a t e , L e g a l , E l e c t r o n i c C o m m e r c e , Small businesses E d u c a t i o n : Re s o u r c e s fo r c o n t e n t a n d D e l i v e r y More recently: Social Networks, Environmental and Scientific Applications, Medicine and Genetics Pe r s o n a l i z e d A p p l i c a t i o n s : b a s e d o n s m a r t m o b i l e d e v i c e s Slide 1- 9 SIMPLIFIED DATABASE SYSTEM ENVIRONMENT Slide 1- 10 Typical DBMS Functionality De f i n e a p a r t i c u l a r d a t a b a s e i n t er m s o f i t s d a t a t y p es , s t r u c t u r es , a n d c o n s t ra i n t s C o n s t r u c t o r L o a d t h e i n i t i a l d a t a b a s e c o n t en t s o n a s ec o n d a r y s t o ra g e m ed i u m M a n i p u l a t i n g t h e d a t a b a s e: Re t r i e va l : Qu e r yi n g , g e n e ra t i n g r e p o r t s Modification: Insertions, deletions and updates to its content A c c e s s i n g t h e d a t a b a s e t h r o u g h We b a p p l i c a t i o n s P roc e s s i n g a n d S h a r i n g b y a s et o f c o n c u r r en t u s er s a n d a p p l i c a t i o n p r o g ra m s – yet , keep i n g a l l d a t a va l i d a n d c o n s i s t en t Slide 1- 11 Application Activities Against a Database A p p l i c a t i o n s i n t e ra c t w i t h a d a t a b a s e b y g e n e ra t i n g - Q u e r i e s : t h a t a c c e s s d i f fe r e n t p a r t s o f d a t a a n d fo r m u l a t e t h e r e s u l t o f a request - Tra n s a c t i o n s : t h a t m a y r e a d s o m e d a t a a n d “ u p d a t e ” c e r t a i n va l u e s o r g e n e ra t e n e w d a t a a n d s t o r e t h a t i n t h e d a t a b a s e Applications must not allow unauthorized users to access data Applications must keep up with changing user requirements against the database Slide 1- 12 Additional DBMS Functionality D BM S m ay a d d itio n a lly p rov id e: Protection or Security measures to prevent unauthorized access “Active” processing to take internal actions on data Presentation and Visualization of data Maintenance of the database and associated programs over the lifetime of the database application – Called database, software, and system maintenance Slide 1- 13 Example of a Database (with a Conceptual Data Model) Mini-world for the example: Part of a UNIVERSITY environment. Some mini-world entities: STUDENTs COURSEs SECTIONs (of COURSEs) (academic) DEPARTMENTs INSTRUCTORs Slide 1- 14 S o m e m i n i -wo r l d r e l a t i on s h i p s : S E CT I ON s a r e o f s p e c i f i c C OU RS E s S T U D E N Ts t a ke S E CT I ON s C OU RS E s h av e p r e r e q u i s i te C OU RS E s I N S T RU CTORs t e a c h S E CT I ON s C OU RS E s a r e o f f e r e d b y D E PA RT M E N Ts S T U D E N Ts m a j o r i n D E PA RT M E N Ts N o t e: T h e a b ove en t i t i es a n d r el a t i o n s h i p s a r e t y p i c a l l y ex p r es s ed i n a c o n c ep t u a l d a t a m o d el , s u c h a s t h e E N T I T Y- R E L AT IO N S HI P d a t a m o d el ( s ee C h a p t er s 3 , 4 ) Example of a Database (with a Conceptual Data Model) Slide 1- 15 EXAMPLE OF A SIMPLE DATABASE Slide 1- 16 Self-describing nature of a database system : A DBMS catalog stores the description of a particular database (e.g. data structures, types, and constraints) The description is called meta-data*. This allows the DBMS software to work with different database applications. Insulation between programs and data: Called program-data independence. Allows changing data structures and storage organization without having to change the DBMS access programs. ----------------------------------------------------------------------------- * Some newer systems such as a few NOSQL systems need no meta-data: they store the data definition within its structure making it self describing Main Characteristics of the Database Approach Slide 1- 17 EXAMPLE OF A SIMPLIFIED DATABASE CATALOG Slide 1- 18 Main Characteristics of the Database Approach (continued) D a ta Ab stra ctio n : A data model is used to hide storage details and present the users with a conceptual view of the database. Programs refer to the data model constructs rather than data storage details S u p p o r t o f m u ltip le v iew s o f th e d a ta : Each user may see a different view of the database, which describes only the data of interest to that user. Slide 1- 19 Main Characteristics of the Database Approach (continued) S h a rin g o f d a ta a n d m u lti-u ser tra n sa ctio n p ro cessin g : Allowing a set of concurrent users to retrieve from and to update the database. Concurrency control within the DBMS guarantees that each transaction is correctly executed or aborted Recovery subsystem ensures each completed transaction has its effect permanently recorded in the database OLTP (Online Transaction Processing) is a major part of database applications. This allows hundreds of concurrent transactions to execute per second. Slide 1- 20 Database Users U sers m ay b e d iv id ed in to Those who actually use and control the database content, and those who design, develop and maintain database applications (called “Actors on the Scene”), and Those who design and develop the DBMS sof tware and related tools, and the computer systems operators (called “Workers Behind the Scene”). Slide 1- 21 Database Users – Actors on the Scene Acto rs o n th e scen e Database administrators: – Re s p o n s i b l e f o r a u th o ri z i n g a c c e s s to th e d a ta b a s e , f o r c o o r d i n a ti n g a n d m o n i to ri n g i ts u s e , a c q u i ri n g s o f tw a r e a n d h a r d w a r e r e s o u r c e s , c o n tr o l l i n g i ts u s e a n d m o n i to ri n g e f f i c i e n c y o f o p e ra t i o n s. Database Designers: – Re s p o n s i b l e to d e f i n e th e c o n te n t, th e s tru c tu r e , th e c o n s tra i n ts , a n d f u n c ti o n s o r t ra n s a c t i o n s a g a i n s t t h e d a t a b a s e. T h ey m u s t c o m m u n i c a t e w i t h t h e e n d- u s e r s a n d u n d e r s ta n d th e i r n e e d s. Slide 1- 22 Database End Users Actors on the scene (continued) End-users: They use the data for queries, reports and some of them update the database content. End-users can be categorized into: – Casual: access database occasionally when needed – Naïve or Parametric: they make up a large section of the end- user population. – They use previously well-defined functions in the form of “c a n n e d t r a n s a c t i o n s ” a g a i n s t t h e d a t a b a s e. – Users of Mobile Apps mostly fall in this category – B a n k- t e l l e r s o r r e s e r v a t i o n c l e r k s a r e p a r a m e t r i c u s e r s w h o d o this activity for an entire shif t of operations. – Social Media Users post and read information from websites Slide 1- 23 Database End Users (continued) – S op h i s t i c a te d : – T h e s e i n c l u d e b u s i n e s s a n a l ys t s , s c i e n t i s t s , e n g i n e e r s , o t h e r s t h o r o u g h l y f a mi l i a r wi t h t h e s ys t e m c a p a b i l i t i e s. – M a ny u s e t o o l s i n t h e f o r m o f s o f t wa r e p a c k a g e s t h a t wo r k c l o s e l y wi t h t h e s t o r e d database. – S t a n d -a l on e : – M o s t l y ma i n t a i n p e r s o n a l d a t a b a s e s u s i n g r e a d y- t o - u s e p a c k a g e d a p p l i c a ti o n s. – A n e x a mp l e i s t h e u s e r o f a t a x p r o g ra m t h a t c r e a t e s i t s o wn i n t e r n a l d a t a b a s e. – A n o t h e r e x a mp l e i s a u s e r t h a t ma i n t a i n s a d a t a b a s e o f p e r s o n a l p h o t o s a n d v i d e o s. Slide 1- 24 Database Users – Actors on the Scene (continued) – Sy s te m An a l y s t s a n d Ap p l i c a t i on De v e l op e rs T h i s c a t e g o r y c u rr e n t l y a c c o u n t s f o r a v e r y l a r g e p r o p o r t i o n o f t h e I T w o rk f o r c e. – Sy s t em An a l y s t s : T h e y u n d e r s t a n d t h e u s e r r e q u i r e me n t s o f n a ï ve a n d s o p h i s t i c a t e d u s e r s a n d d e s i g n a p p l i c a ti o n s i n c l u d i n g c a n n e d tra n s a c ti o n s to me e t th o s e r e q u i r e me n ts. – Ap p l i c a t i o n Pr o g ra m m er s : I mp l e me n t th e s p e c i f i c a ti o n s d e ve l o p e d by a n a l ys ts a n d te s t a n d d e b u g t h e m b e f o r e d e p l o yme n t. – B u s i n es s An a l y s t s : T h e r e i s a n i n c r e a s i n g n e e d f o r s u c h p e o p l e wh o c a n a n a l yz e va s t a mo u n t s o f b u s i n e s s d a t a a n d r e a l - t i me d a t a ( “ Bi g D a t a” ) f o r b e t t e r d e c i s i o n ma k i n g r e l a te d to p l a n n i n g , a d ve r ti s i n g , ma r ke ti n g e tc. Slide 1- 25 Database Users – Actors behind the Scene Tool Developers: Design System Designers and and implement software Implementors: Design systems called tools for Operators and and implement DBMS modeling and designing Maintenance Personnel: packages in the form of databases, performance They manage the actual modules and interfaces and monitoring, prototyping, running and maintenance test and debug them. The test data generation, user of the database system DBMS must interface with interface creation, hardware and software applications, language simulation etc. that environment. compilers, operating facilitate building of system components, etc. applications and allow using database effectively. Slide 1- 26 Advantages of Using the Database Approach Controlling redundancy in data storage and in development and maintenance effor ts. Sharing of data among multiple users. Restricting unauthorized access to data. Only the DBA staff uses privileged commands and facilities. Providing persistent storage for program Objects E.g., Object-oriented DBMSs make program objects persistent– see Chapter 12. Providing Storage Structures (e.g. indexes) for eff icient Quer y Processing – see Chapter 17. Slide 1- 27  PROVIDING PROVIDING PROVIDING REPRESENTING ENFORCING DRAWING OPTIMIZATION OF BACKUP AND MULTIPLE COMPLEX INTEGRITY INFERENCES AND QUERIES FOR RECOVERY INTERFACES TO RELATIONSHIPS CONSTRAINTS ON ACTIONS FROM EFFICIENT SERVICES. DIFFERENT AMONG DATA. THE DATABASE. THE STORED DATA PROCESSING. CLASSES OF USING DEDUCTIVE USERS. AND ACTIVE RULES AND TRIGGERS. Advantages of Using the Database Approach (continued) Slide 1- 28  Potential for enforcing standards: Reduced application development time: This is very crucial for the success of database Incremental time to add each new application is reduced. applications in large organizations. Standards refer to data item names, display formats, screens, report structures, meta-data (description of data), Web page layouts, etc. Additional Implications of Using the Database Approach Slide 1- 29 Additional Implications of Using the Database Approach (continued) F lexib ility to ch a n g e d a ta stru ctu res: Database structure may evolve as new requirements are defined. Ava ila b ility o f cu rren t in fo rm a tio n : Extremely important for on-line transaction systems such as shopping, airline, hotel, car reservations. Eco n o m ies o f sca le: Wasteful overlap of resources and personnel can be avoided by consolidating data and applications across departments. Slide 1- 30 Historical Development of Database Technology Early Database Applications: The Hierarchical and Network Models were introduced in mid 1960 s and dominated during the seventies. A bulk of the worldwide database processing still occurs using these models, par ticularly, the hierarchical model using IBMʼs IMS system. Relational Model based Systems: Relational model was originally introduced in 1970, was heavily researched and experimented within IBM Research and several universities. Relational DBMS Products emerged in the early 1980 s. Slide 1- 31 Historical Development of Database Technology (continued) Object-oriented and emerging applications: Object-Oriented Database Management Systems (OODBMSs) were introduced in late 1980 s and early 1990 s to cater to the need of complex data processing in CAD and other applications. – Their use has not taken off much. Many relational DBMSs have incorporated object database concepts, leading to a new categor y called object- relational DBMSs (ORDBMSs) Extended relational systems add fur ther capabilities (e.g. for multimedia data, text, XML, and other data types) Slide 1- 32 Historical Development of Database Technology (continued) D a t a o n t h e We b a n d E - c o m m e r c e A p p l i c a t i o n s : We b c o n t a i n s d a t a i n H T M L ( H y p e r t e x t m a r k u p l a n g u a g e ) with links among pages. This has given rise to a new set of applications and E- commerce is using new standards like XML (eXtended Markup Language). (see Ch. 13). Script programming languages such as PHP and JavaScript a l l o w g e n e r a t i o n o f d y n a m i c We b p a g e s t h a t a r e p a r t i a l l y generated from a database (see Ch. 11). – A l s o a l l o w d a t a b a s e u p d a t e s t h r o u g h We b p a g e s Slide 1- 33 Extending Database Capabilities (1) N e w f u n c t i o n a l i t y i s b e i n g a d d e d t o DB MS s i n t h e f o l l o wi n g a r e a s : S c i e n t i f i c A p p l i c a t i o n s – P h y s i c s , C h e mi s t r y , B i o l o g y - G e n e t i c s E a r t h a n d A t mo s p h e r i c S c i e n c e s a n d A s t r o n o my X ML ( e X t e n s i b l e Ma r k u p L a n g u a g e ) I ma g e S t o r a g e a n d Ma n a g e me n t A u d i o a n d V i d e o D a t a Ma n a g e me n t D a t a Wa r e h o u s i n g a n d D a t a Mi n i n g – a v e r y ma j o r a r e a f o r f u t u r e d e v e l o p me n t u s i n g n e w t e c h n o l o g i e s ( s e e C h a p t e r s 28-29) S p a t i a l D a t a Ma n a g e me n t a n d L o c a t i o n B a s e d S e r v i c e s T i me S e r i e s a n d Hi s t o r i c a l D a t a Ma n a g e me n t T h e a b o v e g i v e s r i s e t o n e w r e s e a r c h a n d d e v e l o p me n t i n i n c o r p o r a t i n g n e w d a t a t y p e s , c o mp l e x d a t a s t r u c t u r e s , n e w o p e r a t i o n s a n d s t o r a g e a n d i n d e x i n g s c h e me s i n d a t a b a s e s y s t e ms. Slide 1- 34 B a c k g r o u n d s i n c e t h e a d ven t o f t h e 2 1 s t C en t u r y : F i r s t d e c a d e o f t h e 2 1 s t c e n t u r y h a s s e e n t r e me n d o u s g r o wt h i n u s e r g e n e ra t e d d a t a a n d a u t o ma t i c a l l y c o l l e c t e d d a t a f r o m a p p l i c a t i o n s a n d s e a r c h e n g i n e s. S o c i a l M e d i a p l a t f o r ms s u c h a s Fa c e b o o k a n d Twi t t e r a r e g e n e ra t i n g mi l l i o n s o f t ra n s a c t i o n s a d ay a n d b u s i n e s s e s a r e i n t e r e s t e d t o t a p i n t o t h i s d a t a t o “ u n d e r s t a n d ” t h e u s e r s C l o u d S t o ra g e a n d Ba c k u p i s ma k i n g u n l i mi t e d a mo u n t o f s t o ra g e ava i l a b l e t o u s e r s a n d applications Extending Database Capabilities (2) Slide 1- 35 Extending Database Capabilities (3) E m e r g e n c e o f B i g D a t a Te c h n o l o g i e s a n d N O S Q L d a t a b a s e s New data storage, management and analysis technology was necessary to deal with the onslaught of data in petabytes a day (10**15 bytes or 1000 terabytes) in some applications – t h i s s t a r t e d b e i n g c o m m o n l y c a l l e d a s “ B i g D a t a”. H a d o o p ( w h i c h o r i g i n a t e d f r o m Ya h o o ) a n d M a p r e d u c e P r o g r a m m i n g a p p r o a c h t o d i s t r i b u t e d d a t a processing (which originated from Google) as well as the Google file system have given rise to Big Data technologies (Chapter 25). Further enhancements are taking place in the form of Spark b a s e d t e c h n o l o g y. N O S Q L ( N o t O n l y S Q L- w h e r e S Q L i s t h e d e f a c t o s t a n d a r d l a n g u a g e f o r r e l a t i o n a l D B M S s ) systems have been designed for rapid search and retrieval from documents, processing of huge graphs occurring on social networks, and other forms of unstructured data with flexible models of transaction processing (Chapter 24). Slide 1- 36  When not to use a DBMS M a i n i n h i b i t o r s ( c o s t s ) o f u s i n g a DB M S : H i g h i n i t i a l i nve s t me n t a n d p o s s i b l e n e e d f o r a d d i t i o n a l h a r d wa r e. Ove r h e a d f o r p r o v i d i n g g e n e ra l i ty, s e c u r i ty, c o n c u r r e n c y c o n t r o l , r e c o ve r y, a n d i n t e g r i ty f u n c ti o n s. W h en a DB M S m ay b e u n n ec es s a r y : I f th e d a ta b a s e a n d a p p l i c a ti o n s a r e s i mp l e , we l l d e f i n e d , a n d n o t e x p e c te d to c h a n g e. I f a c c e s s to d a ta by mu l ti p l e u s e r s i s n o t r e q u i r e d. W h en a DB M S m ay b e i n fea s i b l e: I n e mb e d d e d s ys te ms wh e r e a g e n e ra l p u r p o s e D BM S may n o t f i t i n ava i l a b l e s to ra g e Slide 1- 37 W h en n o D BM S m ay su ff ice: If there are stringent real-time requirements that may not be met because of DBMS overhead (e.g., telephone switching systems) If the database system is not able to handle the complexity of data because of modeling limitations (e.g., in complex genome and protein databases) If the database users need special operations not supported by the DBMS (e.g., GIS and location based services). When not to use a DBMS Slide 1- 38 Chapter Summary Types of Databases and Database Applications Basic Definitions Typical DBMS Functionality Example of a Database (UNIVERSITY) Main Characteristics of the Database Approach Types of Database Users Advantages of Using the Database Approach Historical Development of Database Technology Extending Database Capabilities When Not to Use Databases Slide 1- 39