OpenMP Programming Training

• Master Trainer program for AICTE
• OpenMP Programming training

Introduction to OpenMP

• OpenMP is an Application Program Interface (API).
• OpenMP provides a portable scalable model for developers of shared memory parallel applications.
• The API supports C, C++, and FORTRAN.
• OpenMP consists of Compiler directives(eg: #pragma), Runtime routines (eg: omp_()), and Environment variables (eg: OMP_).
• The OpenMP Architecture Review Board maintains the OpenMP specification, located at http://www.openmp.org

Threads

• Each process consists of multiple independent instruction streams that are assigned compute resources, and scheduled.
• Threads of a process share the address space.
• Global variables and all dynamically allocated data objects are accessible by all threads of a process.
• Each thread has its own stack, register set, and program counter.
• Threads can communicate by reading/writing variables in the common address space.

Memory Model

• OpenMP is designed for multi-processor/shared memory machines.

Data Handling

• Data can be private or shared.
• Private data is accessed only by its owning thread.
• Shared data is accessible by all threads.
• Data transfer is transparent to the programmer.

OpenMP Stack

• The OpenMP stack includes the user layer, program layer, system layer, and hardware.
• End users interact with the application within the user layer.
• Directives, compilers, OpenMP libraries, and environment variables make up the programming layer.
• The system layer includes the OpenMP runtime library and OS/system support for shared memory and threading.
• The hardware layer consists of processors (Proc1, Proc2, Proc3, ProcN) and shared address space.

OpenMP Programming Model

• OpenMP is a shared memory, thread-based parallelism system.
• OpenMP is based on the existence of multiple threads in the shared memory programming paradigm.
• OpenMP allows explicit parallelism putting the programmer has full control over parallelization.
• Most OpenMP parallelism is specified via compiler directives embedded in the source code.

Fork-Join

• Parallel regions can be nested.

Components of OpenMP

• Compiler Directives: Parallel Construct, Work Sharing, Synchronization, Data Environment (private, first private, last private, shared, reduction)
• Runtime Libraries: Number of threads, Scheduling Type, Nested parallelism, Dynamic Thread Adjustment
• Environment Variables: Number of threads, Thread ID, Dynamic thread adjustment, Nested parallelism

OpenMP Code Structure

• Code is structured as #pragma omp <directive-name> [clause, clause, ...] new-line \
• OpenMP is case sensitive.
• Compiler Directives follow C/C++ standards.
• Only one directive name can be specified per directive.

Parallel Directive

• Creates a team of threads in the parallel region.
• A parallel region is a block of code executed by multiple threads.
• The format is #pragma omp parallel [clause ...] newline if (scalar_expression) private (list) shared (list) firstprivate (list) reduction (operator: list) default (shared | none) copyin (list) num_threads (integer-expression)
• The main thread creates a team of threads and becomes the master of the team.
• The master is a member of that team and has thread ID 0.
• At the start of a parallel region, the code is duplicated, and all threads execute it.
• There is an implied barrier at the end of a parallel section.
• If any thread terminates within a parallel region, all threads in the team terminate.
• A parallel region must be a structured block that does not span multiple routines or code files.
• Only a single IF clause is permitted.
• Only a single NUM_THREADS clause is permitted.

Sequential vs Parallel Code

• Sequential Code Example :

#include <stdio.h>

Void main()
{
    int ID ;
    printf("Hello my ID is : %d\n", ID) ;
}

• Parallel Code Example :

#include <stdio.h>
#include <......>

void main()
{
    <......>
    {
        int ID = <......> ;
        printf(“Hello my ID is : %d\n", ID);
    }
}

Compiler Switches

• GNU Compiler Example: gcc -o omp_helloc -fopenmp omp_hello.c
• Intel Compiler Example: icc -o omp_helloc -fopenmp omp_hello.c

Threads to Spawn

• The number of threads in a parallel region is determined by the following factors, in order of precedence: Evaluation of the IF clause, Setting of the NUM_THREADS clause, Use of the omp_set_num_threads() library function, Setting of the OMP_NUM_THREADS environment variable, Implementation default - usually the number of CPUs, though it could be dynamic.
• Threads are numbered from 0 (master thread) to N-1.

Work Sharing Construct

• Divides the execution of a code region among team members, without launching new threads.
• Restrictions: Must be enclosed within a parallel region, distributes work among threads, is encountered by all threads, and does not launch new threads.
• Types of Work Sharing: for (data parallelism), section (functional parallelism), single (serializes a section of code)

For Directive

• Specifies iterations of the loop immediately following must be executed in parallel by the team.
• Format: #pragma omp for [clause...] newline schedule (type [,chunk]) ordered private (list) firstprivate (list) lastprivate (list) shared (list) reduction (operator: list) nowait
• The below statements are restrictions for the 'for' directive.

for (index = start; index < end; increment_expr)

• It must be possible to determine the number of loop iterations before execution
• No while loops.
• No variations of for loops where the start and end values change.
• Increment must be the same each iteration and it is illegal to branch (goto) out of a loop associated with a for directive

Clauses

• SCHEDULE: Describes how iterations of the loop are divided among the threads in the team.
• The default schedule is implementation dependent.
• Schedule options are static, dynamic, guided, and runtime.
• RUNTIME schedule is determined by the environment variable OMP_SCHEDULE.

Sections Directive

• Each SECTION is executed once by a thread in the team.
• Format:

#pragma omp sections [clause ...] newline
{
    #pragma omp section newline
    structured_block
   
    #pragma omp section newline
    structured_block
}

• NOWAIT implied barrier exists at the end of a SECTIONS directive, unless this clause is used.
• It is illegal to branch into or out of section blocks.
• SECTION directives must occur within the lexical extent of an enclosing SECTIONS directive.

Single Directive

• The enclosed code is to be executed by only one thread in the team.
• May be useful when dealing with sections of code that are not thread safe (such as I/O).
• Format: #pragma omp single [clause ...] newline private (list) firstprivate (list) nowait structured_block