Comparing Parallel Paradigms
- Automatic parallelization combined with explicit Shared Memory programming (compiler directives) used on machines with global memory
- Symmetric Multi-Processors, CC-NUMA, PVP
- These methods collectively known as Shared Memory Programming (SMP)
- SMP programming model works at loop level, and coarse level parallelism:
- the coarse level parallelism has to be specified explicitly
- loop level parallelism can be found by the compiler (implicitly)
-
- Explicit Message Passing Methods are necessary with machines that have no global memory addressability:
- clusters of all sort, NOW & COW
- Message Passing Methods require coarse level parallelism to be scalable
Choosing programming model is largely a matter of the application, personal preference and the target machine.
it has nothing to do with scalability. Scalability limitations:
- communication overhead
- process synchronization
scalability is mainly a function of the hardware and (your) implementation of the parallelism