[Parallel Computing] Intro

Administrivia

Rui Fan
Leshan Wang

curent ddl

score complement

what and why

applications

  1. Fluid dynamics
  2. DNA & drug
  3. Quantum / atomic simulation, cosmological

challenges

  1. Harnessing power of masses
  2. Communication
    1. Processors compute faster than they can communicate.
    2. Problem gets worse as number of processors increase
    3. Main bottleneck to parallel computing
  3. Synchronization
  4. Scheduling
  5. Structured vs. unstructured
    1. Structured problems can be solved with custom hardware.
    2. Unstructured problems more general, but less efficient.
  6. Inherent linmitations
    1. Some problems are not( or don't seem to be ) paralleizable
      1. Dijkstra's shortest paths algorithm
    2. Other problems require clever algorithms to become parallel.
      1. Fibonacci series (\(a_{n}=a_{n-1}+a_{n-2}\))
    3. The human factor
      1. Hard to keep track of concurrent events and dependencies.
      2. Parallel algorithms are hard to design and debug.

Course Outline

  1. Parallel architectures.
    1. shared memory
    2. distributed memory
    3. many more
  2. Parallel languages
    1. OpenMP
    2. MPI
    3. cuda
    4. MapReduce
  3. Algorithm design techniques
    1. Decomposition
    2. Load balancing
    3. schedling
      ## state of the art
  4. parellel computers today mainly based on four processor architectures.
    1. Multi-cores
    2. Many-cores
    3. FPGA
    4. ASIC
  5. power efficiency: goal 50GFLOPS/W
  6. Top-500