Introduction to Parallel and Distributed Computing (SS 2016)
Section outline
-
326.081/326.0AD/326.0PC, Monday 8:30-10:00, S2 120, Start: March 7, 2016
This course is simultaneously offered under the title "Parallel Computing (3KV)"; the detailed organization will be explained in class.
The efficient application of parallel and distributed systems (multi-processors and computer networks) is nowadays an important task for computer scientists and mathematicians. The goal of this course is to provide an integrated view of the various facets of software development on such systems: hardware architectures, programming languages and models, software development tools, software engineering concepts and design patterns, performance modeling and analysis, experimenting and measuring.
Class presentation will be accompanied by hands-on experience on an SGI Altix UltraViolet 1000 distributed shared memory multiprocessor using the following programming models:
- Auto-parallelization and OpenMP and Cilk Plus (Intel C/C++),
- Java multi-threading and networking API,
- Message Programming Interface (MPI-C/C++).
Some background in programming in C/C++ and/or Java is assumed. Course grades will be based on a couple of programming exercises.
To take part in the course, you have to enrol in the KUSSS system. Since the exercises will be submitted via Moodle, you also have to login in Moodle and register as a course participant. You will then also receive per email all messages posted in the News forum.
-
- Introduction
- The SGI UV System
- Semi-Automatic Parallelization and OpenMP and Cilk
- Performance of Parallel Programs
- Multi-Threaded Client/Server Programming
- Parallel Program Design
- The Message Passing Interface MPI
- Distributed Memory Algorithms
Literature
Ian Foster: Designing and Building Parallel Programs -- Concepts and Tools for Parallel Software Engineering, Addison Wesley, Reading, MA, 1995. Amazon page.Peter Pacheco: An Introduction to Parallel Programming. Morgan Kaufmann, 2011. Amazon Page.Michael J. Quinn: Parallel Programming in C with with MPI and OpenMP, McGraw-Hill, New York, 2003. Source code, Amazon page.
Kai Hwang: Advanced Computer Architecture: Parallelism, Scalability, Programmability, McGraw-Hill, New York, 1993. Amazon page.Restricted Material
The password to this section is handed out in class. -
A quick cross-section of the course.
-
Basic concepts and their application on the SGI UV; material partially taken from the SGI developer documentation.
- Automatic Parallelism (Compiler Output)
- Automatic Parallelization Example
- OpenMP Example
- Cilk Plus Example
- Programming Documentation
-
Modeling and analyzing the performance of parallel programs, some material from Chapter 3 of Foster's Designing and Building Parallel Programs.
- Presentation
- Figures (restricted)
-
Developing Internet-based parallel programs by simple language/operating system mechanisms.
- Alle Fäden in der Hand (in German)
- Java multi-threading and networking API
- Numerische Aufgaben mit Java lösen (restricted)
- Matrix Multiplication (MatMult.java, MatMultThreads.java, MatMultNet.java)
- All Pairs Shortest Paths (PathJ.java, PathThread.java, PathThreadPool.java)
-
The engineering of parallel programs based on Chapter 2 of Foster's Designing and Building Parallel Programs.
- Presentation
- Figures (restricted)
-
Programming in MPI based on Chapter 8 of Foster's Designing and Building Parallel Programs.
-
- Presentation
- Figures (restricted)
-
Four programming/benchmarking exercises will be handed out which will be used for grading. For getting credited with the KV3 "Parallel Computing" course, additionally a small team project is to be elaborated.