OpenMP Programming Model

• Multi Processing Open specifications are used by OpenMP
• Application Program Interface (API) explicitly directs multi-threaded, shared memory parallelism
• The three primary API components are compiler directives, runtime library routines, and environment variables
• The API is specified for C, C++, and Fortran
• OpenMP is portable and easy to use

OpenMP Programming Model

• OpenMP is designed for multi-processor/core, shared memory machines
• OpenMP programs achieve parallelism through the use of threads
• A thread of execution is the smallest processing unit scheduled by an operating system
• Threads exist within a single process and cease to exist without it
• Typically, the number of threads matches the number of processor cores but can differ
• OpenMP is an explicit (not automatic) programming model giving the programmer control
• Most OpenMP parallelism is specified via compiler directives in C/C++ or Fortran source code

Fork-Join Model

• OpenMP programs begin as a single process with the master thread
• The master thread executes sequentially until a parallel region construct is encountered
• FORK: The master thread creates a team of parallel threads, becomes their master, and assigned thread number 0
• JOIN: Team threads synchronize and terminate after completing the parallel region construct, leaving the master thread

OpenMP API Overview

• The three API components are Compiler Directives, Runtime Library Routines, and Environment Variables
• Compiler Directives guide compilers and appear as source code comments
• The OpenMP API includes an increasing number of run-time library routines
• OpenMP provides environment variables to control parallel code execution at run-time
• OpenMP core syntax focuses on compiler directives
• Most constructs in OpenMP are compiler directives: #pragma omp (construct name) [clause [clause]...]
• OpenMP code structure is a structured block, with one point of entry and one point of exit
• Function prototypes and types can be found in the file: #include <omp.h)

Compiling and Running OpenMP Programs

• Example Multi-threaded C program that prints "Hello World!". (hello_omp.c)
• To compile the 'hello_omp.c' program using gcc, use the command: gcc -fopenmp hello_omp.c -o hello_omp
• To execute the compiled 'hello_omp' program, type ./hello_omp

Parallel Region Construct

• A parallel region is a code block executed by multiple threads, the fundamental OpenMP parallel construct
• Syntax is #pragma omp parallel [clause[[,] clause] ... ] structured block

Typical Clauses in a Clause List Include

• Degree of concurrency using 'num_threads()'
• Data scoping includes 'private()', 'firstprivate()', 'shared()', and 'default()'
• If () determines whether the parallel construct creates threads
• Code example demonstrating an OpenMP parallel directive

Creating Threads

• Threads can be created in OpenMP using the parallel construct
• Creating a 4-thread parallel region uses the num_threads clause(Example 1)
• Creating a 4-thread parallel region uses runtime library routine(Example 2)
• Ways to set threads in a parallel region are by runtime library function omp_set_num_threads()
• Setting the 'num_threads' e.g., #pragma omp parallel num_threads (8), or at runtime via export OMP_NUM_THREADS=8

Computing PI

• Numerical integration 1∫0 4.0/(1+x^2) = π can be used to compute the PI
• Develop a serial program for the problem
• Parallelize the serial program with the OpenMP directive
• Compare the obtained results
• An example of a serial program is displayed for computing PI
• 'Method I' is one approach for parallel program execution.

False Sharing

• If independent data elements sit on the same cache line, updates cause 'slosh back and forth' between threads i.e. false sharing
• Processors execute operations faster than data access in memory
• A fast memory block (cache) is added to a processor
• Cache design considers temporal and spatial locality
• When x is shared (x = 5), and my_y and my_z are private, their value will depend on which thread executes first

Cache Coherence

• When caches of multiple processors store the same variable, an update by one processor is 'seen' by all
• If threads with separate caches access variables in the same cache line, updating a variable invalidates the cache line
• Other threads then retrieve values from main memory
• Threads don't share (only a cache line), which behaves like sharing a variable, called false sharing

Race Condition

• Race conditions occur when multiple threads update a shared variable resulting in non-deterministic computation
• Only one thread can update shared resources at a time

Synchronization

• Brings one or more threads to a defined point in execution
• Barrier and mutual exclusion are synchronization forms
• Barrier: Each thread waits until all threads arrive
• Mutual exclusion: A code block that only one thread at a time can execute
• Synchronization imposes order constraints and protects access to shared data
• Critical sections in Method II of computing (π) apply this technique
• Critical section - block of code by multiple threads to updates the shared variable, by only one thread at a time

High Level Synchronizations

• A critical construct is mutual exclusion, allows only one specific thread at a time
• An 'atomic' construct, is a basic form that provides mutual exclusion but only applies to the memory location update
• A 'barrier' construct makes each thread waits for the arrival of all threads

Summary Points

• OpenMP execution model details the parallel region in OpenMP programs
• The following constructs are used parallel, critical, atomic and barrier
• OpenMP uses library functions and utilizes environment variables.
• False sharing and race condition are known issues in OpenMP programs.