Introduction to Parallel and Distributed Computing (SS 2020)

Section outline

• Select section Introduction to Parallel and Distributed Computing (SS 2020)

  Introduction to Parallel and Distributed Computing (SS 2020)

  Collapse all Expand all

  326.081/326.0AD, Monday 8:30-10:00, S2 120, Start: March 2, 2020
  

  The efficient application of parallel and distributed systems (multi-processors and computer networks) is nowadays an important task for computer scientists and mathematicians. The goal of this course is to provide an integrated view of the various facets of software development on such systems: hardware architectures, programming languages and models, software development tools, software engineering concepts and design patterns, performance modeling and analysis, experimenting and measuring.

  Class presentation will be accompanied by hands-on experience on an SGI Altix UltraViolet 1000 distributed shared memory multiprocessor using the following programming models:

  • Auto-parallelization and OpenMP (Intel C/C++),
  • Java multi-threading and networking API,
  • Message Programming Interface (MPI-C/C++).

  Some background in programming in C/C++ and/or Java is assumed. Course grades will be based on a couple of programming exercises.
  

  To take part in the course, you have to enrol in the KUSSS system. Since the exercises will be submitted via Moodle, you also have to login in Moodle and register as a course participant. You will then also receive per email all messages posted in the News forum.

  • Select activity Announcements

    

    Announcements Forum
  • Select activity Questions and Answers

    

    Questions and Answers Forum
  • Select activity Zoom Meetings: Monday 10:00-10:15 https://jku.zoo...

    Zoom Meetings: Monday 10:00-10:15
    https://jku.zoom.us/j/133230027?pwd=V3VLbW5UTDU0QnZ2Z0d2d3FXM1ZwZz09
    Meeting-ID: 133 230 027
    Password: intropar
    

• Select section Introduction and Organization

  Introduction and Organization

  • Select activity Introduction to Parallel and Distributed Computing

    

    Introduction to Parallel and Distributed Computing File
  • Select activity How to Use the Parallel Computing Server

    

    How to Use the Parallel Computing Server Page
  • Select activity How to Use the Server From Other Networks (restricted)

    

    How to Use the Server From Other Networks (restricted) URL
  • Select activity Some More Information (in German)

    

    Some More Information (in German) Page
  • Select activity Restricted Area (password handed out in class)

    

    Restricted Area (password handed out in class) URL
• Select section Parallel Architectures

  Parallel Architectures

  • Select activity Video "SGI UV" (Lecture March 9/16)

    

    Video "SGI UV" (Lecture March 9/16) URL
  • Select activity Parallel Architectures

    

    Parallel Architectures File
  • Select activity Technical Advances in the SGI UV Architecture

    

    Technical Advances in the SGI UV Architecture File
  • Select activity Performance and Productivity Breakthroughs with Very Large Coherent Shared Memory: The SGI ® UV Architecture

    

    Performance and Productivity Breakthroughs with Very Large Coherent Shared Memory: The SGI ® UV Architecture File
  • Select activity SGI UV Datasheet

    

    SGI UV Datasheet File
  • Select activity JKU Datasheet

    

    JKU Datasheet File
  • Select activity The Supercomputer "MACH-2"

    

    The Supercomputer "MACH-2" URL
• Select section Auto-Parallelization and OpenMP

  Auto-Parallelization and OpenMP

  • Select activity Video "Automatic Parallelization 1" (Lecture March 16)

    

    Video "Automatic Parallelization 1" (Lecture March 16) URL
  • Select activity Video "Automatic Parallelization 2" (Lecture March 16/23)

    

    Video "Automatic Parallelization 2" (Lecture March 16/23) URL
  • Select activity Automatic Parallelism

    

    Automatic Parallelism File
  • Select activity Intel® C++ Compiler Developer Guide and Reference

    

    Intel® C++ Compiler Developer Guide and Reference URL
  • Select activity Automatic Parallelization

    

    Automatic Parallelization URL
  • Select activity Example

    

    Example File
  • Select activity matmult.c

    

    matmult.c File
  • Select activity Video "OpenMP" (Lecture March 30)

    

    Video "OpenMP" (Lecture March 30) URL
  • Select activity OpenMP

    

    OpenMP File
  • Select activity Sources

    

    Sources File
  • Select activity Example

    

    Example File
  • Select activity OpenMP

    

    OpenMP URL
  • Select activity OpenMP Specification

    

    OpenMP Specification URL
  • Select activity OpenMP Tutorial

    

    OpenMP Tutorial URL
  • Select activity Intel OpenMP* Support

    

    Intel OpenMP* Support URL
  • Select activity OpenMP* Pragmas Summary

    

    OpenMP* Pragmas Summary URL
  • Select activity Worksharing Using OpenMP*

    

    Worksharing Using OpenMP* URL
  • Select activity Thread Affinity Interface

    

    Thread Affinity Interface URL
  • Select activity GNU libgomp

    

    GNU libgomp URL
  • Select activity Enabling OpenMP

    

    Enabling OpenMP URL
  • Select activity GOMP_CPU_AFFINITY

    

    GOMP_CPU_AFFINITY URL
• Select section Performance Analysis

  Performance Analysis

  Modeling and analyzing the performance of parallel programs, some material from Chapter 3 of Foster's Designing and Building Parallel Programs.
  

  • Select activity Video "Performance Analysis" (Lecture March 23)

    

    Video "Performance Analysis" (Lecture March 23) URL
  • Select activity Performance Analysis

    

    Performance Analysis File
• Select section Multi-Threaded Client/Server Programming

  Multi-Threaded Client/Server Programming

  • Select activity Video "Concurrency in Java" (Lecture April 20)

    

    Video "Concurrency in Java" (Lecture April 20) URL
  • Select activity Concurrency in Java

    

    Concurrency in Java File
  • Select activity Sources

    

    Sources File
  • Select activity Lesson Concurrency

    

    Lesson Concurrency URL
  • Select activity Trail: Custom Networking

    

    Trail: Custom Networking URL
  • Select activity Posix Threads Programming

    

    Posix Threads Programming URL
  • Select activity C Sockets

    

    C Sockets URL
  • Select activity Client Example

    

    Client Example URL
  • Select activity Server Example

    

    Server Example URL
• Select section Parallel Program Design

  Parallel Program Design

  The engineering of parallel programs based on Chapter 2 of Foster's Designing and Building Parallel Programs.
  

  • Select activity Video "Parallel Program Design" (Lecture April 27/May 11)

    

    Video "Parallel Program Design" (Lecture April 27/May 11) URL
  • Select activity Parallel Program Design

    

    Parallel Program Design File
• Select section The Message Passing Interface MPI

  The Message Passing Interface MPI

  Programming in MPI, partially based on Chapter 8 of Foster's Designing and Building Parallel Programs.
  

  • Select activity Video "MPI" (Lecture May 18/25)

    

    Video "MPI" (Lecture May 18/25) URL
  • Select activity MPI

    

    MPI File
  • Select activity Sources

    

    Sources File
  • Select activity MPI Forum

    

    MPI Forum URL
  • Select activity MPI Specification

    

    MPI Specification URL
  • Select activity MPI Tutorial

    

    MPI Tutorial URL
• Select section Distributed Memory Algorithms

  Distributed Memory Algorithms

  • Select activity Video "Distributed Memory Algorithms" (Lecture June 8)

    

    Video "Distributed Memory Algorithms" (Lecture June 8) URL
  • Select activity Presentation

    

    Presentation File
  • Select activity Figures (restricted)

    

    Figures (restricted) URL
• Select section Exercises

  Exercises

  Four programming/benchmarking exercises will be handed out. The best three results will be used for grading.
  

  • Select activity Exercise 1 (April 20)

    

    Exercise 1 (April 20) Assignment
  • Select activity Exercise 2 (May 11)

    

    Exercise 2 (May 11) Assignment
  • Select activity Exercise 3 (June 1)

    

    Exercise 3 (June 1) Assignment
  • Select activity Exercise 4 (June 29)

    

    Exercise 4 (June 29) Assignment