[Parallel Computing] Shared Memory Programming and OpenMP

image-20200330122725049

shared memory model compared with mpi

  • Threads (e.g. Pthreads, Java)
    • individual sequences of instructions that can execute in parallel and access shared data
    • very general, because programmer have to manage everything.
  • parallel programming language/ library
    • a parallel language or library is used to create code
    • require new compiler
  • Compiler directives (OMP)
    • inserts compiler directives into a sequential program to specify parallelism and indicate shared data and the compiler translates into threads.
    • still uses threads underneath, but system manages the threads.
    • easy to program. but lose flexibility.

Processes and thread

  • Process(MPI)
    • separate program with its own variable, memory, stack and instruction pointer.
    • Different programs can't access each other's memory

    image-20200330123249520

  • thread(OMP)
    • Concurrent routine that shares the variables and memory space, but has its own stack and instruction pointer.

    image-20200330123330196