Applications of Parallel Computers
FIT-PDDAcad. year: 2020/2021
The course gives an overview of usable parallel platforms and models of programming, mainly shared-memory programming (OpenMP), message passing (MPI) and data-parallel programming (CUDA, OpenCL). A parallelization methodology is completed by performance studies and applied to a particular problem. The emphasis is put on practical aspects and implementation.
Learning outcomes of the course unit
To learn how to parallelize various classes of problems and predict their performance. To be able to utilize parallelism and communication at thread- and process level. To get acquainted with state-of-the-art standard interfaces, language extensions and other tools for parallel programming (MPI, OpenMP). Write and debug a parallel program for a selected task.
Parallel architectures with distributed and shared memory, programming in C/C++ with MPI and OpenMP, GPGPU, parallelization of basic numerical methods.
Types of parallel computers, programming in C/C++, basic numerical methods
- recommended prerequisite
Recommended optional programme components
Recommended or required reading
David Patterson John Hennessy: Computer Organization and Design MIPS Edition, Morgan Kaufmann, 2013, p. 800, ISBN: 978-0-12-407726-3
Victor Eijkhout: Introduction to High Performance Scientific Computing, 2015, 978-1257992546
Kirk, D., and Hwu, W.: Programming Massively Parallel Processors: A Hands-on Approach, Elsevier, 2017, p. 540, ISBN: 978-0-12-811986-0
Planned learning activities and teaching methods
Assesment methods and criteria linked to learning outcomes
Language of instruction
To clarify possibilities of parallel programming on multi-core processors, on clusters and on GP GPUs. View over synchronization and communication techniques. Get to know a method of parallelization and performance prediction of selected real-world applications, the design of correct programs and the use parallel computing in practice.
Topics for the state doctoral exam (SDZ):
- Indicators and laws of parallel processing, function of constant efficiency, scalability.
- Parallel processing in OpenMP, SPMD, loops, sections, tasks and synchronization primitives.
- Architectures with shared memory, UMA, NUMA, cache coherence protocols.
- Blocking and non-blocking pair-wise communications in MPI.
- Collective communications in MPI, parallel input-output.
- Architecture of superscalar processors, algorithms for out-of-order instruction execution.
- Data parallelism SIMD and SIMT, HW implementation and SW support.
- Architecture of graphics processing units, differences from superscalar CPUs.
- Programming language CUDA, thread and memory models..
Specification of controlled education, way of implementation and compensation for absences
Defence of a software project based on a variant of parallel programming.
Type of course unit
39 hours, optionally
Teacher / Lecturer
- Parallel computer architectures, performance measures and their prediction
- Patterns for parallel programming
- Synchronization and communication techniques.
- Shared variable programming with OpenMP
- Message-passing programming with MPI
- Data parallel programming with CUDA/OpenCL
- Examples of task parallelization and parallel applications
Guided consultation in combined form of studies
26 hours, optionally
Teacher / Lecturer