Strumenti Utente

Strumenti Sito


magistraleinformaticanetworking:spd:lezioni14.15

Questa è una vecchia versione del documento!


Journal of Lessons, SPD year 2014-2015

Journal

  • 24/02/2015 Course introduction : parallel programming frameworks and high-level approach to parallel programming over different platforms: MPI, TBB and OpenCL as main examples; course organization and prerequisites; reference books and studying material. MPI (Message Passing Interface) standard : brief history and aim of the standard, single program / multiple data execution model, compilation and linkage model; issues in supporting multiple programming languages and uses (application, utility library and programming language support) with a static compilation and linkage approach. Portability in parallel programming: functional and non-functional aspects, performance tuning and performance debugging. MPI as a parallel framework that supports a structured approach to parallel programming. Basic concepts of MPI.
  • 25/02/2015 MPI basic concepts : communicators (definition, purpose, difference between inter and intra-communicators, process ranks); point to point communication (concepts of envelope, local/global completion, blocking/non-blocking primitive, send modes); collective communications (definition, communication scope, global serialization, freedom of implementation in the standard); MPI datatypes (basic meaning and use, primitive / derived datatypes, relationship with sequential language types).
  • 03/03/2015 MPI : point to point communication semantics (buffer behaviour, receive, wildcards, status objects), MPI datatypes (purpose as explicitly defined meta-data provided to the MPI implementation, semantics, typemap and type signature, matching rules for communication, possible data conversions in heterogeneous distributed architectures, role in MPI-performed packing and unpacking, examples); basic and derived datatypes (multiple language bindings, code-instantiated metadata, examples)
  • 04/03/2015 MPI : core primitives for datatype creation (MPI_Type_* : contiguous, vector, hvector, indexed, hindexed, struct, commit, free) and examples; point to point communication modes (MPI_RSend usage); non-blocking communication (Wait and Test group of primitives, semantics, MPI_Request object handles to active requests); communicators and groups (communicator design aim and programming abstraction, local and global information, attributes and virtual topologies, groups as local objects, primitives for locally creating and managing groups)
  • 10/03/2015 Lesson is postponed to 12/03/2015 (see NEWS on the course page).
  • 11/03/2015 MPI : canceling and testing cancellation of non-blocking primitives (issues and pitfalls, interaction with MPI implementation); MPI_finalize; intracommunicators (basic primitives concerning size, rank, comparison; communicator creation as a collective operation, MPI_Comm_create basic and general case, MPI_Comm_split; rationale of use of communicator creation and duplication as a tool to create isolate communication spaces for subprograms/skeletons and for parallel application libraries; impact on performance)
  • 12/03/2015 MPI Lab. Writing structured MPI programs, point to point communications (ping pong, token ring and variants) and how to write reusable code by exploiting communicators, first examples with derived datatypes (indexed, vector and their combinations).
  • 17/03/2015 MPI Lab. Design and implementation of a simple farm skeleton in MPI. Reusability and separation of concerns in MPI; exploiting communicators for skeleton and inside skeleton implementation (examples with farm extensions: worker initialization, worker status collection at the end of the stream, work stealing); different communication primitives (Synch/Buffered and Blocking/non blocking) wrt farm load distributions strategies: Round Robin, Job request, implicit job request with double buffering.
  • 18/03/2015 MPI collective communications (definition and semantics, execution environment, basic features, agreement of key parameters among the processes, constraints on Datatypes and typemaps for collective op.s, overall serialization vs synchronization, potential deadlocks); taxonomy of MPI collectives (blocking/non-blocking, synchronization/communication/communication+computation, asymmetry of the communication pattern, variable size versions, all- versions); MPI_IN_PLACE and collective op.s.; basic blocking collective operations (barrier, broadcast, gather/gatherV, scatter/scatterV, allgather, alltoall/alltoallv).
  • 24/03/2015 MPI Compute and communication collectives, MPI_Reduce, semantics; MPI Operators (arithmetic, logic, bitwise, MINLOC and MAXLOC) and their interaction with Datatypes; defining MPI custom operators via MPI_Create_op.
  • 25/03/2015 MPI Lab. Adding generic support for farm workers' state and reinitialization from the emitter (streams and substreams), semantics and design choices. MPI General Reduce and Scan operators: Allreduce, Reduce_scatter_block, Reduce_scatter, Scan, Exscan.
  • 31/03/2015 Thead Building Blocks (TBB) Thread Building Blocks C++ template library overview: purpose, abstraction mechanisms and implementation layers (templates, runtime, supported abstractions); tasks vs threads and parallel patters composability; parallel_for, ranges and partitioners; task partitioning and scheduling, grain size and affinity; quick survey of use of lambda expression, containers and mutexes.
  • 01/04/2015 MPI IO Rationale of MPI-IO; basic concepts (file, file handle, etype, filetype, view); setting a file view; file size, offset and file pointer (shared, individual); file open, MPI_Info and most important tags; primitives for basic file management (close, delete, set and get size); examples.
  • 15/04/2015
  • 23/04/2015 TBB laboratory Simple Mandelbrot set example, use of parallel for and blocked range, speedup on 8 cores.
  • 28/04/2015 KDD examples Short introduction to Knowledge Discovery in Databases and Data Mining; examples; parallelism exploitation in data mining algorithms; K-means algorithm.
  • 29/04/2015 TBB laboratory Implementation of K-means with TBB, from example code to a running program; farm (parallel for) and reduce based parallelization of the inner loop. Tuning the TBB program for speedup.
  • 05/05/2015 Project discussion Overall structure of the final project.
  • 06/05/2015
  • 12/05/2015
  • 13/05/2015
  • 19/05/2015
  • 20/05/2015
  • 26/05/2015

Slides, Notes and References to papers

magistraleinformaticanetworking/spd/lezioni14.15.1431418278.txt.gz · Ultima modifica: 12/05/2015 alle 08:11 (9 anni fa) da Massimo Coppola