Hands-on; Shared Memory III; MPI with Pthread, OpenMP

1. Computes the dot product of two vectors;
   • http://siber.cankaya.edu.tr/ozdogan/ParallelComputing/cfiles/code51.c Serial - Serial program.
   • http://siber.cankaya.edu.tr/ozdogan/ParallelComputing/cfiles/code52.c Pthreads only - A shared memory programming model using Pthreads.
   • http://siber.cankaya.edu.tr/ozdogan/ParallelComputing/cfiles/code53.c MPI only - A distributed memory programming model with MPI.
   • http://siber.cankaya.edu.tr/ozdogan/ParallelComputing/cfiles/code54.c MPI with pthreads - A hybrid model that utilizes both MPI and Pthreads to execute on systems that are comprised of clusters of SMP's.
   • http://siber.cankaya.edu.tr/ozdogan/ParallelComputing/cfiles/makecode5154 Makefile - Use as

      make -f makecode5154
     

   In general, there may be problems if multiple threads make MPI calls. The program may fail or behave unexpectedly. If MPI calls must be made from within a thread, they should be made only by one thread.
2. Another example (http://siber.cankaya.edu.tr/ozdogan/ParallelComputing/cfiles/code55.c program) for MPI with Pthread.
3. OpenMP. Two example programs are given; http://siber.cankaya.edu.tr/ozdogan/ParallelComputing/cfiles/code25.c hello, http://siber.cankaya.edu.tr/ozdogan/ParallelComputing/cfiles/code26.c workshare. Follow the steps below before executing OpenMP codes;

   export PGI=/usr/local/pgi
   export PATH=$PGI/linux86/6.2/bin:$PATH
   export MANPATH=$MANPATH:$PGI/linux86/6.2/man
   export LD_LIBRARY_PATH=/usr/local/pgi/linux86/6.2/liblf:
   /usr/local/pgi/linux86/6.2/lib:$LD_LIBRARY_PATH
   

4. Hello world
   • In this simple example, the master thread forks a parallel region.
   • All threads in the team obtain their unique thread number and print it.
   • The master thread only prints the total number of threads. Two OpenMP library routines are used to obtain the number of threads and each thread's number.

     export OMP_NUM_THREADS=4
     pgcc -o omp_hello code25.c -mp
     

5. Loop work-sharing
   • The iterations of a loop are scheduled dynamically across the team of threads.
   • A thread will perform CHUNK iterations at a time before being scheduled for the next CHUNK of work.

     pgcc -o omp_workshare code26.c -mp