Three layer cake for shared-memory programming

Abstract

There are many different styles of parallel programming for shared-memory hardware. Each style has strengths, but can conflict with other styles. How can we use a variety of these styles in one program and minimize their conflict and maximize performance, readability, and flexibility? This paper surveys the relative advantages and disadvantages of three styles (SIMD, fork join, and message passing), shows how to compose them hierarchically, and advises how to choose what goes at each level in the hierarchy.