Introduction to Parallel and Distributed Computing (SS 2022)

Section outline

• Select section Introduction to Parallel and Distributed Computing (SS 2022)

  Introduction to Parallel and Distributed Computing (SS 2022)

  Collapse all Expand all

  326.081/326.0AD, Monday 8:30-10:00, Room: S2 Z74, Start: March 7, 2022
  

  The efficient application of parallel and distributed systems (multi-processors and computer networks) is nowadays an important task for computer scientists and mathematicians. The goal of this course is to provide an integrated view of the various facets of software development on such systems: hardware architectures, programming languages and models, software development tools, software engineering concepts and design patterns, performance modeling and analysis, experimenting and measuring.

  Class presentation will be accompanied by hands-on experience on an SGI Altix UltraViolet 1000 distributed shared memory multiprocessor using the following programming models:

  • Auto-parallelization and OpenMP (Intel C/C++),
  • Java multi-threading and networking API,
  • Message Programming Interface (MPI-C/C++).

  Some background in programming in C/C++ and/or Java is assumed. Course grades will be based on a couple of programming exercises.
  

  To participate in the course, you have to register in the KUSSS System. Since the assignments will be uploaded in Moodle, you also have to create an account in the RISC Moodle (press the button "Create new account") and self-enrol in the course (press the button "Enrol me"); then you also receive all messages posted in the course forums per email.
• Select section Videos

  Videos

  As an extra service, it is intended to live-stream and record the lectures via the following Zoom meeting:

  https://jku.zoom.us/j/92654875045?pwd=d2lYdk9XV1hqcFBzajh2YXlsYVhjUT09
  Meeting-ID: 926 5487 5045 Password: intropar

  However, no guarantee is given with respect to completeness and quality of the stream/recordings. The basic course format is on-site, not hybrid.

  • Select activity Video Lecture March 7, 2022

    

    Video Lecture March 7, 2022 URL
  • Select activity Video Lecture March 14, 2022

    

    Video Lecture March 14, 2022 URL
  • Select activity Video "Performance Analysis" (substitute for Lecture of March 21, 2022)

    

    Video "Performance Analysis" (substitute for Lecture of March 21, 2022) URL
  • Select activity Video Lecture March 28, 2022

    

    Video Lecture March 28, 2022 URL
  • Select activity Video Lecture April 4, 2022

    

    Video Lecture April 4, 2022 URL
  • Select activity Video Lecture April 25, 2022

    

    Video Lecture April 25, 2022 URL
  • Select activity Video Lecture May 2, 2022

    

    Video Lecture May 2, 2022 URL
  • Select activity Video Lecture May 9, 2022

    

    Video Lecture May 9, 2022 URL
  • Select activity Video Lecture May 16, 2022

    

    Video Lecture May 16, 2022 URL
  • Select activity Video Lecture May 23, 2022

    

    Video Lecture May 23, 2022 URL
  • Select activity Video Lecture May 30, 2022

    

    Video Lecture May 30, 2022 URL
• Select section Introduction and Organization

  Introduction and Organization

  • Select activity Introduction to Parallel and Distributed Computing

    

    Introduction to Parallel and Distributed Computing File
  • Select activity How to Use the Parallel Computing Server

    

    How to Use the Parallel Computing Server Page
  • Select activity How to Use the Server From Other Networks (restricted)

    

    How to Use the Server From Other Networks (restricted) URL
  • Select activity Some More Information (in German)

    

    Some More Information (in German) Page
  • Select activity Restricted Area (password handed out in class)

    

    Restricted Area (password handed out in class) URL
• Select section Parallel Architectures

  Parallel Architectures

  • Select activity Parallel Architectures

    

    Parallel Architectures File
  • Select activity Technical Advances in the SGI UV Architecture

    

    Technical Advances in the SGI UV Architecture File
  • Select activity Performance and Productivity Breakthroughs with Very Large Coherent Shared Memory: The SGI ® UV Architecture

    

    Performance and Productivity Breakthroughs with Very Large Coherent Shared Memory: The SGI ® UV Architecture File
  • Select activity SGI UV Datasheet

    

    SGI UV Datasheet File
  • Select activity JKU Datasheet

    

    JKU Datasheet File
  • Select activity The Supercomputer "MACH-2"

    

    The Supercomputer "MACH-2" URL
• Select section Auto-Parallelization and OpenMP

  Auto-Parallelization and OpenMP

  • Select activity Automatic Parallelism

    

    Automatic Parallelism File
  • Select activity Intel® C++ Compiler Classic Developer Guide and Reference

    

    Intel® C++ Compiler Classic Developer Guide and Reference URL
  • Select activity Automatic Parallelization

    

    Automatic Parallelization URL
  • Select activity Example

    

    Example File
  • Select activity matmult.c

    

    matmult.c File
  • Select activity OpenMP

    

    OpenMP File
  • Select activity Sources

    

    Sources File
  • Select activity Example

    

    Example File
  • Select activity OpenMP

    

    OpenMP URL
  • Select activity OpenMP Specification

    

    OpenMP Specification URL
  • Select activity OpenMP Tutorial

    

    OpenMP Tutorial URL
  • Select activity Intel OpenMP* Support

    

    Intel OpenMP* Support URL
  • Select activity OpenMP* Pragmas Summary

    

    OpenMP* Pragmas Summary URL
  • Select activity Worksharing Using OpenMP*

    

    Worksharing Using OpenMP* URL
  • Select activity Thread Affinity Interface

    

    Thread Affinity Interface URL
  • Select activity GNU libgomp

    

    GNU libgomp URL
  • Select activity Enabling OpenMP

    

    Enabling OpenMP URL
  • Select activity GOMP_CPU_AFFINITY

    

    GOMP_CPU_AFFINITY URL
• Select section Performance Analysis

  Performance Analysis

  Modeling and analyzing the performance of parallel programs, some material from Chapter 3 of Foster's Designing and Building Parallel Programs.
  

  • Select activity Performance Analysis

    

    Performance Analysis File
• Select section Multi-Threaded Client/Server Programming

  Multi-Threaded Client/Server Programming

  • Select activity Concurrency in Java

    

    Concurrency in Java File
  • Select activity Sources

    

    Sources File
  • Select activity Lesson Concurrency

    

    Lesson Concurrency URL
  • Select activity Trail: Custom Networking

    

    Trail: Custom Networking URL
  • Select activity Posix Threads Programming

    

    Posix Threads Programming URL
  • Select activity C Sockets

    

    C Sockets URL
  • Select activity Client Example

    

    Client Example URL
  • Select activity Server Example

    

    Server Example URL
• Select section Parallel Program Design

  Parallel Program Design

  The engineering of parallel programs based on Chapter 2 of Foster's Designing and Building Parallel Programs.
  

  • Select activity Parallel Program Design

    

    Parallel Program Design File
• Select section The Message Passing Interface MPI

  The Message Passing Interface MPI

  Programming in MPI, partially based on Chapter 8 of Foster's Designing and Building Parallel Programs.
  

  • Select activity MPI

    

    MPI File
  • Select activity Sources

    

    Sources File
  • Select activity MPI Forum

    

    MPI Forum URL
  • Select activity MPI Specification

    

    MPI Specification URL
  • Select activity MPI Tutorial

    

    MPI Tutorial URL
• Select section Distributed Memory Algorithms

  Distributed Memory Algorithms

  • Select activity Presentation

    

    Presentation File
  • Select activity Figures (restricted)

    

    Figures (restricted) URL
• Select section Exercises

  Exercises

  Four programming/benchmarking exercises will be handed out. The best three results will be used for grading.
  

  • Select activity Exercise 1 (April 25)

    

    Exercise 1 (April 25) Assignment

    Due: Monday, 25 April 2022, 11:59 PM
  • Select activity Exercise 2 (May 16)

    

    Exercise 2 (May 16) Assignment

    Due: Monday, 16 May 2022, 11:59 PM
  • Select activity Exercise 3 (June 6)

    

    Exercise 3 (June 6) Assignment

    Due: Monday, 6 June 2022, 11:59 PM
  • Select activity Exercise 4 (July 4)

    

    Exercise 4 (July 4) Assignment

    Due: Monday, 4 July 2022, 11:59 PM