Working on HPC clusters
About the course | Intended audience | Prerequisites | Content details | Scheduled events
About the course
A practical introduction to High Performance Computing (HPC) for researchers working with computationally demanding analyses and large datasets.
High Performance Computing systems are widely used in fields such as bioinformatics, machine learning, image analysis and large-scale data processing. This course introduces the core concepts needed to work effectively on HPC systems, including accessing remote servers, managing files, running analyses and scheduling jobs using SLURM.
Participants learn how HPC systems are organised, how to interact with them using the Unix command line and how to develop efficient workflows for computational research.
By the end of the course, participants should be able to:
- describe what an HPC system is and how it differs from a standard computer
- distinguish between login nodes and compute nodes
- connect to and work on remote HPC systems using the command line
- transfer files to and from HPC systems using graphical and command-line tools
- understand the role of job schedulers and computational resource management
- submit and monitor jobs using the SLURM scheduler
- use SLURM job arrays to parallelise similar analyses efficiently
- apply HPC workflows to their own computational research projects
The course is highly practical, combining presentations, demonstrations and hands-on exercises throughout.
Back to topIntended audience
This course is suitable for:
- researchers and students who need to run computationally intensive analyses
- participants working with bioinformatics, machine learning, image analysis or large datasets
- researchers interested in learning how to use remote Linux servers and HPC systems
- participants who currently run demanding analyses on personal computers and want to scale their workflows
- researchers who have completed introductory Unix training and want to extend those skills to HPC environments
Prerequisites
A working knowledge of the Unix command line is essential before attending this course.
Participants should already be comfortable navigating the filesystem and running basic shell commands in a Unix/Linux environment.
If you are new to Unix command-line computing, we strongly recommend completing an introductory Unix course or working through equivalent training materials before attending.
Content details
The course combines lectures, demonstrations and hands-on practical exercises focused on real-world HPC workflows.
Topics covered include:
- Introduction to High Performance Computing
Overview of HPC systems, including how they are organised and why they are used for computationally intensive research tasks such as bioinformatics, machine learning and large-scale data analysis.
- Accessing and navigating HPC systems
Connecting to remote Linux servers, understanding filesystem organisation and working effectively in command-line HPC environments.
- Login nodes and compute nodes
Understanding the roles of different components within an HPC cluster and learning best practices for interactive work and large-scale computation.
- File transfer and remote development workflows
Moving files between local machines and remote systems using tools such as FileZilla and command-line utilities, together with editing files directly on remote systems using Visual Studio Code.
- Running jobs with SLURM
Introduction to job scheduling using the SLURM workload manager, including submitting, monitoring and managing jobs on shared computing infrastructure.
- Parallel workflows and job arrays
Using SLURM job arrays and resource customisation to parallelise similar analyses efficiently and scale computational workflows for larger datasets.
The course also includes an optional introduction to the free HPC facilities available at the University of Cambridge, although the core material is broadly applicable to HPC systems used in many research environments.
Topics include High Performance Computing (HPC), remote servers, shell scripting and SLURM-based workflows.