Introduction to Parallel and Distributed Computing (SS 2014)
Abschnittsübersicht
-
Monday, 8:30-10:00, T 642, Start: Mach 3, 2014
The efficient application of parallel and distributed systems (multi-processors and computer networks) is nowadays an important task for computer scientists and mathematicians. The goal of this course is to provide an integrated view of the various facets of software development on such systems: hardware architectures, programming languages and models, software development tools, software engineering concepts and design patterns, performance modeling and analysis, experimenting and measuring.
Class presentation will be accompanied by hands-on experience on an SGI Altix 4700 distributed shared memory multiprocessor using the following programming models:
- Auto-parallelization and OpenMP (Intel C/C++ or Fortran),
- Java multi-threading and networking API,
- Message Programming Interface (MPI-C/C++ or Fortran).
Some background in programming in C/C++ and/or Fortran and/or Java is assumed. Course grades will be based on a couple of programming exercises.
To take part in the course, you have to enrol in the KUSSS system. Since the exercises will be submitted via Moodle, you also have to login in Moodle and register as a course participant. You will then also receive per email all messages posted in the News forum.
-
- Introduction
- The SGI Altix System
- Semi-Automatic Parallelization
- Performance of Parallel Programs
- Multi-Threaded Client/Server Programming
- Parallel Program Design
- The Message Passing Interface MPI
- Distributed Memory Algorithms
Literature
Ian Foster: Designing and Building Parallel Programs -- Concepts and Tools for Parallel Software Engineering, Addison Wesley, Reading, MA, 1995. Amazon page.Peter Pacheco: An Introduction to Parallel Programming. Morgan Kaufmann, 2011. Amazon Page.Michael J. Quinn: Parallel Programming in C with with MPI and OpenMP, McGraw-Hill, New York, 2003. Source code, Amazon page.
Kai Hwang: Advanced Computer Architecture: Parallelism, Scalability, Programmability, McGraw-Hill, New York, 1993. Amazon page.Restricted Material
The password to this section is handed out in class. -
A quick cross-section of the course.
-
Architecture and operating system.
-
Basic concepts and their application on the Altix; material partially taken from the SGI developer documentation.
- Automatic Parallelism (Compiler Output)
- Automatic Parallelization Example
- OpenMP Example
- Linux Application Tuning Guide
- SGI Altix Applications Development and Optimization (restricted)
- Programming Documentation
-
Modeling and analyzing the performance of parallel programs, some material from Chapter 3 of Foster's Designing and Building Parallel Programs.
- Presentation
- Figures (restricted)
-
Developing Internet-based parallel programs by simple language/operating system mechanisms.
- Alle Fäden in der Hand (in German)
- Thread-Level Parallelism (Section from "Topics in IRIX Programming")
- Sockets - The GNU C Library
- Java multi-threading and networking API
- Numerische Aufgaben mit Java lösen (restricted)
-
The engineering of parallel programs based on Chapter 2 of Foster's Designing and Building Parallel Programs.
- Presentation
- Figures (restricted)
-
Programming in MPI based on Chapter 8 of Foster's Designing and Building Parallel Programs.
-
- Presentation
- Figures (restricted)
-
Four programming/benchmarking exercises will be handed out which will be used for grading.