Introduction to Databases

Introduction to Databases

• Databases are essential for managing data efficiently in today's digital world
• Used across various industries, including e-commerce, social media, banking, healthcare, education, logistics, CRM, and government services

Before Databases

• Information was previously stored using methods like paper, magnetic tapes, books, and electronic files
• These methods suffered from limitations such as difficulty searching, retrieving, and managing large data volumes, lacking integrity and security

Evolution of Databases

• 1970s: The Entity-Relationship (ER) model became a standard tool for database design
• Oracle introduced the first Relational Database Management System (RDBMS)
• 1980s: SQL became the standard language, and more companies (like Sybase and early Microsoft SQL Server) began creating RDBMSs

1990s/2000s Developments

• NoSQL databases emerged to manage unstructured data (images, text, audio)
• Data Warehousing and data mining became prominent
• Big Data and cloud-based solutions emerged

Basic Concepts of Databases

• A database is organized data stored in tables with rows and columns
• Tables contain related data, enabling efficient access and management
• Databases can store information about people, products, orders, and more

Database Management Systems (DBMS)

• DBMS is software to manage databases
• It creates, maintains, and retrieves data
• It handles creating, updating, deleting, and querying data
• It acts as an intermediary between users and the database, ensuring security and data integrity (e.g., Oracle Database, Microsoft SQL Server, MySql)

DBM Tools and Software

• Applications that interact with data
• Used to create, manage, and manipulate databases
• Provide tools to query, update and report on data
• Features include large-volume storage, retrieval using languages such as SQL and data manipulation via insert, update, and delete options

The 3 Architecture Levels in a Database

• Conceptual design defines entities and their relationships, a high level abstraction
• Logical level specifies tables, columns and relationships, abstract but detailed
• Physical level describes how and where data is stored, lowest level of abstraction

Relational Databases

• Data is organized into tables with rows and columns
• Tables are linked through relationships (created by primary and foreign keys)
• Relationships improve data integrity and efficiency, preventing data redundancy

Entity Relationship Diagrams (ERDs)

• Diagrams used to show entities, attributes (fields) and their relationships in a database
• Helps visualize relationships graphically

Normalization

• Process to improve data integrity and efficiency
• Reduces data redundancy
• Optimizes data storage and retrieval
• Uses normal forms (1NF, 2NF, 3NF) which are different design rules

SQL and its Importance

• Structured Query Language (SQL) is used to manage and manipulate relational databases
• Essential in data analysis and extracting meaningful insights
• Enables users to perform queries to extract insights from datasets

Data Types

• Different data types are used to store different kinds of data (integers, floating-point, character strings, dates)

Data Structures

• Tables are the fundamental structural unit for storing and managing data in databases

Data Analysis with Queries

• Queries (SQL statements) are used to retrieve data, aggregate data, and filter results
• Includes sorting, grouping, and using arithmetic and logical operators

Joining Tables

• Uses in queries to combine rows from tables with matching columns

PowerBI as a Database Tool

• A data visualization and business intelligence tool
• Can be used to create and manage databases, through PowerQuery
• Also use AI tools (e.g., Copilot) to automate tasks
• Primarily works on a desktop or through a cloud-based platform

Big Data

• Large amounts of data that a system can handle.
• Characterized by volume, velocity, variety, variability and value
• Data is obtained from a mixture of structured, unstructured and semi-structured sources (social media, IoT, transactions, sensors).

NoSQL Databases

• Not based on relational model
• Designed to handle different data types (e.g., structured, semi-structured data, unstructured data)
• Designed to handle a changing data model more easily than relational databases
• Offer horizontal scalability because their design enables them to store and manage large volumes of data across multiple servers
• Popular types include document stores (MongoDB), key-value stores (Redis), graph databases (Neo4j)

Database Storage Methods

• Hadoop Distributed File System (HDFS): handles extremely large amounts of data across multiple servers.
• Data lakes are centralized repositories for various types of data that is stored as received, ready for usage at any point in time.

Economic and Financials Data Sources

• Multiple sources provide financial and economic data

Explore the fundamentals of databases, their historical evolution, and the significant advancements in database technologies. This quiz covers the transition from traditional data storage methods to modern relational and NoSQL databases, highlighting key developments from the 1970s to the 2000s.