Data Storage: Physical vs Logical

• Item 1 introduces data storage concepts.
• It covers physical vs logical approaches.
• Physical and logical structures are important.
• Databases and storage paradigms are explored.

Information & Data

• Information Technology is the scientific knowledge and techniques enabling the automatic processing of information by computers.
• Information is knowledge communication/acquisition that expands or clarifies current knowledge.
• Data is information adequately prepared for being processed by a computer. From Latin "datum", meaning "that is given".

Information Characteristics

• Information is communicated through perceptible physical events tied to conventional codification.
• Characteristics of storage include:
  • Perdurability: How long the information persists.
  • Capacity: Amount of information stored.
  • Speed: Access time.
  • Range: Accessibility to multiple receivers/locations.
  • Access Type: Privileged, handy, or external.

Types of Data Storage

• Main storage is agile, provides immediate access, and is often privileged, but has capacity/cost limitations, short range and is volatile.
• Secondary storage is slow, requires external access, but offers high capacity/cost, broad range and persistence.
• Main storage suits immediate term PROCESSING.
• Secondary storage focus is medium-long term STORING.

Storage Media Chronology

• This shows the timeframes for these different media
• SSD
• Optical Disc
• Magnetic Disk
• Hard Drive
• Magnetic Tape
• Punched Tape

Media, Devices, & Files

• Media is the material capable of recording information physically.
• A Storage Device is media with a mechanism (hardware) for providing storage (recording and retrieving).
• A File is an information container within the device, a subset of recorded infos with its own organization grouped under a label within a non-volatile device.
• Archive is each information container from the users' point of view

Data Storage Scopes

• Logical scope refers to how clients and users see the data.
• Physical or internal scope refers to how the devices store the data.
• App (SGBD), OS (FMS)

Data Structures

• Data structures relate logical and physical views
• Relate cars and persons with differing cardinality

Basic Data Operations

• Users need to search, add, edit, retrieve and erase.
• Devices are able to locate, write and read

Data Essentials

• Semantics (user coding) and timeliness are needed.
• Effectiveness relates to purpose, while efficiency relates to timeliness

Basic Goals

• Basic goals are effectiveness (efficacy) and efficiency.
• Effectiveness is solving the valid problem.
• Efficiency is to minimize consumption.

Physical Structures

• In a storehouse, actions include incorporating, removing, modifying, and looking up but locating is required to incorporate, remove, modify and look up

Serial Organization

• "Disorganized" elements
• Optimal Insertion & Space Use
• Requires a hard locating process (full scan).

Sequential Organization

• Sorted Elements
• Improves selection process "by a key".
• Rest of selection processes not improved.
• Hinders actualizations.
• Degenerates and require maintenance.

Hash Organization

• Elements located "in their place".
• Optimal selection by 'a key'.
• Hinders selection by any other key.
• Wastes lots of space.
• May require maintenance.

Indexed Organization

• 'Pointed' elements.
• Reduced selection cost.
• Increased actualization cost.
• Auxiliary Oranization.
• May require maintenance.

Course Planning

• The course is split into halves with different weightings and evaluations.
• The first half is ~62% weight and involves two compulsory labworks, one theoretical test CA, and one exercise in the finals.
• The second half is ~38% weight and includes one compulsory labwork, one theoretical test (final), and one exercise in the final.

The Value of Information

• For statics, restriction brings value.
• For dynamics, efficiency brings value.

Storage Types

• Data storages are classified into:
  • Structured: follows a pre-established distribution with semantics and value, requires design, characterized by its definition (strongly typed files). Examples are forms, cards, and tables
  • Non-structured: Lacks structure or pattern. Examples are (free) plain text, photos, audio/video files.
  • Semi-structured: Combination of the former two, examples are minutes or legal documents

ANSI/SPARC Architecture

• Proposed in 1975 by the 'Standards Planning and Requirements Committee' of the 'American National Standards Institute frames databases with three levels:
• Internal Level: Physically stored records described by an Internal Schema. Data - devices, for administrators.
• Conceptual Level: Global view of data structures described by a Conceptual Schema. Data - Data", for designers.
• External Level: User view of database described by an External Schema. Data - users, for users.

Data Model

• The idea is to obtain the discourse universe properties (from clients/users, in natural language) and describe them through a 'data structure' (formal language).
• Properties of the Discourse Universe consider both statics (invariant in time) which uses structures and dynamics (variant in time) using modeling operators.

Data Model Statics

• Data Model Stats focus on objects, associations and restrictions.
• Restrictions are constraints put on the Database (inherent with model structure, semantic with the data).
• Semantic constraints follow from semantic assumptions
• Semantic constrains ensure the integrity of the base and validity of content

Concept of Database

• Collection or storage of integrated data with controlled redundancy.
• Structures reflects interrelations and restriction from the real world, the data is independent from the application and users.
• It has a unique definition and description and stored with them, integrated.
• Procedures preserve integrity while observing availability and confidentiality.

Databases and DBMS Approach

• Archive Schemas and File Systems: -Storage approach focuses on processes.
• Database Systems: -Storage approach focuses on data.

DBMS Tools

• A coordinated set of tools supporting interaction with the DB at any level.
• Tools include programs, procedures, languages.
• Interaction with the DB describes, stores, retrieves, and handles data.
• Interaction with the DB provides integrity, confidentiality, and security.
• Works at any level with users, programmers, analysts, designers, and admins.

DBMS Essential Functions

• Description: Defines data elements and their structure, and interrelations between them, plus validation rules.
• Manipulation: Must support operation of the Database content.
• Utilization: Should include a set of tools for ensuring use and support administration.

Evolution of the Storage

• Data storage evolution determined by:
  • Hardware: cheaper and faster media, enhanced communications.
  • Needs: by institutions, companies, and markets for efficient solutions.
  • Research: models/tools proposed before needs/hardware exist
    • the Relational Model

Paradigms - Evolution

• As technology, use, and needs evolve, so do the functionalities supporting the organization.
• Data Files: area-specific applications at the operative level.
• Data Bases (late '60): data/use independence, +integrity, -redundancy, +versatility, +value for OLTP.
• Data Warehouse: Synthesize history data, decision-making support for OLAP.

Paradigms - OLTP

• On-Line Transaction Processing provides services for the database operataion processing.
• There should be Difference between interactive operations (on-line) and batch.
• A transaction:
  • Atomicity: Completed or will not take place.
  • Consistence: Preserves integrity rules.
  • Isolation: Operations are independent.
  • Durability: Permanent effects upon transaction commitment.
• Structured Databases: essential at the institution's operative level, Featuring robustness and availability. Usually set on client-server arch.
• Structured Data Models like Relational Model, hierarchical and network

Paradigms - OLAP

• On-Line Analytic Processing seeks efficiency in analytic queries with huge volumes.
  • ROLAP: built on relational engine (de-normalized tables)
  • MOLAP: multidimensional databases
  • HOLAP: hybrid DB (part relational, part multidimensional)
  • Goals include include: aggregation, comparison, correlation, classification, prediction, simulation, etc.
• Data sources: from own structured DB (data warehousing). Can be weak (or absent) logical structures + simple physical structures Huge volume + structureless + simple organization = hard working feasible through distribution and massive parallelism

DBMS Evolution

• Navigational DBMS: (60s) -Physical/relative pointers, Hierarchical/Network data models, used during the 70's and efficient with conventional DB
• Relational DBMS (70's) -Logical/symbolic pointers, Relational Model used since early 80's and is effective/accessible.
• Massive (structured) Databases (80/90s) -Data Warehousing, object oriented or document oriented
• NoSQL: (00') -No structure, no model, no limit - Big Data, Perspectives oriented to analytic needs
• CloudDB: (10') -NewSQL/CloudDB/XaaS supported
• Next?: -Full Semantic support: RDF/Open/Linked data