Distributed Systems and Cloud Computing: Message Passing Interface (MPI) Core Functions

MPI Core Functions

• MPI_Init: initializes the MPI library, must be the first routine called
• MPI_Comm_size: gets the size of a communicator
• MPI_Comm_rank: gets the rank of the calling process in the communicator
• MPI_Send: sends a message to another process
• MPI_Recv: receives a message from another process
• MPI_Finalize: cleans up all MPI state, must be the last MPI function called by a process

MPI Communication Functions

Blocking Communication

• Completion of the call is dependent on certain events (e.g., data sent or safely copied to system buffer space)
• Functions: MPI_Send, MPI_Recv
• Status object provides information about: source process, message tag, error status
• MPI_Get_count returns the number of elements received

Non-blocking Communication

• Communication routine returns without waiting for completion
• Programmer's responsibility to ensure buffer is free for reuse
• Functions: MPI_Isend, MPI_Irecv
• Used to increase performance by overlapping computation with communication

Data Movement (Collective Communication)

• MPI_Bcast: broadcasts a message from the process with rank "root" to all other processes in the group
• MPI_Scatter: splits the message into n equal segments and sends each segment to a different process
• MPI_Gather: collects data from all processes in the group and sends it to a single process
• MPI_Alltoall: performs an all-to-all communication where every process sends and receives n data segments
• MPI_Alltoallv: a generalization of MPI_Alltoall where each process sends/receives a customizable amount of data

Broadcasting with MPI

• MPI_Bcast: sends the same data to all processes in a communicator
• Used for sending user input to a parallel program or configuration parameters to all processes
• MPI_Bcast function: MPI_Bcast(void* data, int count, MPI_Datatype datatype, int sender, MPI_Comm communicator)

Synchronization (Collective Communication)

• MPI_Barrier: causes each process to block until all tasks in the group reach the same MPI_Barrier call
• Used for synchronization between processes

Reductions (Collective Computation)

• MPI_Reduce: collects data from the other members and performs an operation (min, max, add, multiply, etc.) on that data
• Examples of operations: MPI_MAX, MPI_MIN, MPI_SUM, MPI_PROD, etc.

Test your understanding of the core functions in Message Passing Interface (MPI) used in Distributed Systems and Cloud Computing. This quiz covers the essential MPI functions, including MPI_Init, MPI_Comm_size, and MPI_Comm_rank.