Höchstleistungsrechenzentrum Stuttgart: BOOTCAMP: Multi-GPU Programming

Prerequisites and content levels

Prerequisites

Experience with C/C++, NVIDIA CUDA C++, and MPI knowledge is required.

Content levels

Intermediate level: 3:30 hours
Advanced level: 6:30 hours

Learn more about course curricula and content levels.

Instructors

Main lecturers: Paul Graham (NVIDIA)

Agenda

- preliminary -

All times in CEST:

Day 0 (Mon, June 16) : Cluster Dry Run

11:00 - 12:00 Cluster Dry Run Session

Day 1 (Tue, June 17): Parallelizing a Single GPU Code

09:00 - 09:05 Welcome
09:05 - 09:30 Introduction to Nsight Systems and GPU Computing (Lecture and Read-only Lab)
09:30 - 10:30 Overview of a Single GPU Code
10:30 - 12:30 Parallelizing a Single GPU Code using CUDA memcpy, Streams, and an Introduction to Single Node Multi-GPU ( P2P ) (Lecture and Lab)
12:30 - 13:00 Lunch
13:00 - 15:00 Introduction to Multi-Node Multi-GPU (GPU Direct ) programming (Lecture and Lab)

Day 2 (Wed, June 18): Introduction to NCCL and NVSHMEM

09:00 - 11:00 Introduction to NVIDIA Collectives Communications Library (NCCL) (Lecture and Lab)
11:00 - 13:00 Introduction to NVSHMEM (Lecture and Lab)
13:00 - 13:30 Demo of the Nsight Systems Profiler Report for a Multi-Node Application

Hands-on sessions

Attendees will be given access to a GPU cluster for the duration of the Bootcamp.

The code is publicly available on github.

Registration information

Registration will be open soon.

The final participants will be selected and informed after the registration deadline has passed.

This course is offered in cooperation by HLRS, JSC, LRZ, VSC Vienna, Cyfronet AGH, IT4Innovations, RISE, LiU, OpenACC.org and NVIDIA.

Registration closes on May 9, 2025 at 23:55.

Fees

Students without Master's degree or equivalent. Participants from EU or EuroCC countries only: 0 EUR
PhD students or employees at a German university or public research institute: 0 EUR
PhD students or employees at a university or public research institute in an EU or EuroCC country other than Germany: 0 EUR.
Other participants, e.g., from industry, other public service providers, or government. Participants from EU or EuroCC countries only: 0 EUR

Our course fee includes coffee breaks (in classroom courses only).

For lists of EU and EuroCC countries have a look at the Horizon Europe and EuroCC website.

Only participants from institutions belonging to these countries can take part in this course.

Contact

Maksym Deliyergiyev phone 0711 685 87261, training(at)hlrs.de

HLRS Training Collaborations in HPC

HLRS is part of the Gauss Centre for Supercomputing (GCS), together with JSC in Jülich and LRZ in Garching near Munich. EuroCC@GCS is the German National Competence Centre (NCC) for High-Performance Computing. HLRS is also a member of the Baden-Württemberg initiative bwHPC.

This course is provided within the framework of EuroCC2.

Acknowledgements

Within the scope of the EuroCC2 project this training is offered free of charge. By participating in a training free of charge, companies receive state-aid corresponding to the regular market-prices listed in the service portfolio. Please note state-aid regulations in Germany.

EuroCC 2 and EuroCC4SEE have received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 101101903 and No 101191697. The JU receives support from the European Union’s Digital Europe Programme and Germany, Bulgaria, Austria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, Greece, Hungary, Ireland, Italy, Lithuania, Latvia, Poland, Portugal, Romania, Slovenia, Spain, Sweden, France, Netherlands, Belgium, Luxembourg, Slovakia, Norway, Türkiye, Republic of North Macedonia, Iceland, Montenegro, Serbia, Bosnia and Herzegovina.

Official course URL(s)

http://www.hlrs.de/training/2025/BC-MULTIGPU-NV.

Further courses

See the training overview and the Supercomputing Academy pages.