Course detail

Practical Parallel Programming

FIT-PPPAcad. year: 2019/2020

The course covers architecture and programming of parallel systems with functional and data parallelism. First, the parallel system theory and program parallelization are discussed. The detailed description of most proliferated supercomputing systems, interconnection network typologies and routing algorithms is followed by the architecture of parallel and distributed storage systems. The course goes on in message passing programming in standardized interface MPI. Consequently, techniques for parallel debugging and profiling are discussed. Last part of the course is devoted to the description of parallel programming patterns and case studies from the are of linear algebra, physical systems described by partial differential equations, N-Body systems and Monte-Carlo methods.

Supervisor

doc. Ing. Jiří Jaroš, Ph.D.

Department

Department of Computer Systems (UPSY)

Learning outcomes of the course unit

Overview of principles of current parallel system design and of interconnection networks, communication techniques and algorithms. Survey of parallelization techniques of fundamental scientific problems, knowledge of parallel programming in MPI. Knowledge of basic parallel programming patterns. Practical experience with the work on supercomputers, ability to identify performance issues and propose their solution.
Knowledge of capabilities and limitations of parallel processing, ability to estimate performance of parallel applications. Language means for process/thread communication and synchronization. Competence in hardware-software platforms for high-performance computing and simulations.

Prerequisites

Von-Neumann computer architecture, computer memory hierarchy, cache memories and their organization, programming in assembly and in C/C++. Knowledge gained in courses PRL and AVS.

Co-requisites

Not applicable.

Recommended optional programme components

Not applicable.

Recommended or required reading

aktuální PPT prezentace přednášek
http://mpitutorial.com/
http://www.cs.kent.edu/~jbaker/ParallelProg-Sp11/
Pacecho, P.: Introduction to Parallel Programming. Morgan Kaufman Publishers, 2011, 392 s., ISBN: 9780123742605
https://pages.tacc.utexas.edu/~eijkhout/pcse/html/
Hennessy, J.L., Patterson, D.A.: Computer Architecture - A Quantitative Approach. 5. vydání, Morgan Kaufman Publishers, Inc., 2012, 912 s., ISBN: 9780123747501

Planned learning activities and teaching methods

Not applicable.

Assesment methods and criteria linked to learning outcomes

Assessment of a project, 10 hours in total and a midterm examination.
Exam prerequisites:
To get 20 out of 40 points for projects and midterm examination.

Language of instruction

Czech

Work placements

Not applicable.

Aims

To get familiar with the architecture of distributed supercomputing systems, their interconnection networks and storage. To orientate oneself in parallel systems on the market, be able to assess communication and computing possibilities of a particular architecture and to predict the performance of parallel applications. Learn how to write portable programs using standardized interfaces and languages, specify parallelism and process communication. To learn how to practically use supercoputer for solving complex engineering problems.

Specification of controlled education, way of implementation and compensation for absences

Missed labs can be substituted in alternative dates (monday or friday)
There will be a place for missed labs in the last week of the semester.

Classification of course in study plans

Programme IT-MGR-2 Master's
branch MBI , any year of study, summer semester, 5 credits, compulsory-optional
branch MGM , any year of study, summer semester, 5 credits, compulsory-optional
branch MIS , any year of study, summer semester, 5 credits, elective
branch MBS , any year of study, summer semester, 5 credits, elective
branch MIN , any year of study, summer semester, 5 credits, elective
branch MMM , any year of study, summer semester, 5 credits, elective
Programme MITAI Master's
specialization NADE , any year of study, summer semester, 5 credits, elective
specialization NBIO , any year of study, summer semester, 5 credits, compulsory
specialization NGRI , any year of study, summer semester, 5 credits, elective
specialization NNET , any year of study, summer semester, 5 credits, elective
specialization NVIZ , any year of study, summer semester, 5 credits, elective
specialization NCPS , any year of study, summer semester, 5 credits, elective
specialization NSEC , any year of study, summer semester, 5 credits, elective
specialization NISD , any year of study, summer semester, 5 credits, elective
specialization NIDE , any year of study, summer semester, 5 credits, elective
specialization NISY , any year of study, summer semester, 5 credits, elective
specialization NMAL , any year of study, summer semester, 5 credits, elective
specialization NMAT , any year of study, summer semester, 5 credits, elective
specialization NSEN , any year of study, summer semester, 5 credits, elective
specialization NVER , any year of study, summer semester, 5 credits, elective
specialization NSPE , any year of study, summer semester, 5 credits, elective
Programme IT-MGR-2 Master's
branch MPV , 1. year of study, summer semester, 5 credits, compulsory
branch MSK , 1. year of study, summer semester, 5 credits, compulsory
Programme MITAI Master's
specialization NHPC , 1. year of study, summer semester, 5 credits, compulsory
specialization NEMB , 2. year of study, summer semester, 5 credits, compulsory

Type of course unit

Lecture

26 hours, optionally

Teacher / Lecturer

doc. Ing. Jiří Jaroš, Ph.D.

Syllabus

Introduction to parallel processing.
Architectures with distributed memory,
Interconnection networks: topology and routing algorithms, switching, flow control.
Technologies of interconnection networks (Infiniband).
Distributed file systems (Lustre, HPFS).
Message passing interface, pair-wise communications, data types
Collective communications and communicators,
Hybrid programming OpenMP/MPI and one-sided communications.
Parallel code debugging, profiling and tracing.
Programming patterns for parallel programming.
Case studies: matrix calculations, linear equation systems
Case studies: solution of PDE systems, finite difference, spectral methods
Case studies: Fluid dynamics, N-Body systems, Monte-Carlo.

Exercise in computer lab

16 hours, compulsory

Teacher / Lecturer

doc. Ing. Jiří Jaroš, Ph.D.

Syllabus