Лекция 1: Параллельное программирование

Lecture 1: Generations

• First generation (1940-1956): Vacuum tubes
• Second generation (1956-1963): Transistors
• Third generation (1964-1971): Integrated circuits
• Fourth generation (1971-present): Microprocessors
• Fifth generation (present and beyond): Artificial intelligence (AI)

Lecture 1: Basic Parallel Models

• SISD (Single Instruction, Single Data): Uniprocessors (e.g., Pentium)
• SIMD (Single Instruction, Multiple Data): Vector processors
• MIMD (Multiple Instruction, Multiple Data): Multiprocessor architecture
• MISD (Multiple Instruction, Single Data): Doesn't commonly exist

Lecture 1: Architectures

• Shared memory
• Distributed memory
• Hybrid

Lecture 1: Processes and Threads

• Process: An instance of the operating system (OS) to execute a program.
• Thread: The smallest unit of processing; a sequence of instructions. Threads share global static data, heap (dynamic data), and stack (local data).

Lecture 1: Parallel Programming Models

• Message passing: Parallel processes exchange data by sending messages. Examples include PVM and MPI.
• Shared memory: Parallel threads share a global address space. Examples include POSIX threads and OpenMP.

Lecture 2: MPI Terminology - Basics

• MPI (Message Passing Interface): A task is an instance, a sub-program, or a process in an MPI program.
• Group: An ordered set of process identifiers.
• Rank: A unique number assigned to each task.
• Context: A property that partitions the communication space.
• Communicator: The scope for communication operations.

Lecture 2: MPI Functions

• Initialization and management calls
• Point-to-point communication
• Collective operations
• Data type creation

Lecture 2: Speed-up and Scalability

• Strong scaling: Problem size stays constant.
• Weak scaling: Problem size scales proportionally with the number of processors.
• Amdahl's Law: A formula that models the speedup that can be achieved by parallelism.

Lecture 3: Blocking/Non-blocking Communication

• Blocking communication: Waits for the communication operation to complete.
• Non-blocking communication: Allows the program to continue while the communication operation is in progress.

Lecture 3: Parts of Messages

• Data part (BUF, COUNT, DATATYPE)
• Message envelope (SOURCE, DEST, TAG, COMM)

Lecture 3: Communication Modes (Send modes)

• Synchronous send: Completes only when the receive has started.
• Buffered send: Always completes, regardless of whether the receive has started, by copying the message.
• Standard send: Chooses between Ssend, Bsend based on internal buffer.
• Ready send: Sends bit-by-bit once the receive is ready.

Lecture 3: Communication Modes (Receive modes)

• Receive call for all modes.
• Wildcards are used for receiving messages. Examples are MPI_ANY_SOURCE and MPI_ANY_TAG.

Lecture 3: Deadlock

• A situation where processes are waiting for sends or receives that can never occur.

Lecture 4: Communication Modes

• Non-blocking communication is aimed at overlapping communication and computations.

Lecture 5: Collective Communication

• Characteristics: Synchronization may or may not occur. No use of tags.
• Receive/send buffers must have the same size. Terminating all MPI processes associated with a communicator.

Lecture 6: Global Reduction Operators

• Associative: Order of sub-reductions not specified.
• Examples include max, min, sum, product, logical AND/OR, bitwise AND/OR.

Lecture 7-8: Only Syntax (No details provided in the text)

Lecture 9: Topologies

• Topologies are how processes are connected.
• Independent of the actual hardware network.
• Essential for structure-generic libraries and variable graph structures.

Lecture 10: Graph Topology

• Uses hierarchical systems.
• Communicators can be intra (within a group) or inter (between groups).
• Default communicators: MPI_COMM_WORLD, MPI_COMM_SELF.

Lecture 11: Derived Datatypes

• Derived from basic datatypes using constructors.
• MPI datatypes are opaque.
• Defined by: a list of basic datatypes, and displacements (positive, zero, or negative).

Этот тест охватывает различные аспекты параллельного программирования, включая поколения вычислительных систем, модели параллельной обработки и архитектуры. Также рассматриваются процессы и потоки, а также их взаимодействие в средах с общей и распределенной памятью.