Home Quantum ESPRESSO on exascale.mahidol.ac.th Multi-Node-Multi-GPUs MNMG Testing
Post
Cancel

Quantum ESPRESSO on exascale.mahidol.ac.th Multi-Node-Multi-GPUs MNMG Testing

Quantum ESPRESSO is an integrated suite of Open-Source computer codes for electronic-structure calculations and materials modeling at the nanoscale based on density-functional theory, plane waves, and pseudopotentials.

What can PWscf do ?

PWscf performs many different kinds of self-consistent calculations of electronic-structure properties within Density-Functional Theory (DFT), using a Plane-Wave (PW) basis set and pseudopotentials (PP).[1]

In particular:

  • ground-state energy and one-electron (Kohn-Sham) orbitals, atomic forces, stresses;

  • structural optimization, also with variable cell;

  • molecular dynamics on the Born-Oppenheimer surface, also with variable cell;

  • macroscopic polarization (and orbital magnetization) via Berry Phases;

  • various forms of finite electric fields, with a sawtooth potential or with the modern theory of polarization;

  • Effective Screening Medium (ESM) method

Download Dataset:

The environment variable BENCHMARK_DIR will be used throughout the example to refer to the directory containing the AUSURF112 input files.

1
2
3
4
5
6
cd ~
mdkir qe
mkdir qe/ausurf
cd qe/ausurf
wget https://repository.prace-ri.eu/git/UEABS/ueabs/-/raw/master/quantum_espresso/test_cases/small/Au.pbe-nd-van.UPF
wget https://repository.prace-ri.eu/git/UEABS/ueabs/-/raw/master/quantum_espresso/test_cases/small/ausurf.in

Run QE on One Compute Node One GPU

As suggestion rules for runing on GPU QE on HPC and GPU systems:

  • 1 GPU : 1 MPI
  • Fill the CPU with OpenMP threads
  • No task groups (-ntg 1)
  • No parallel eigensolver (-ndiag 1), no diagonized?
1
2
3
4
5
$ cd ~/qe/ausurf

$ module unload *
$ module load quantum_espresso/v7.0
$ mpirun -n 1  pw.x -npool 1 -ntg 1 -ndiag 1 -input ausurf.in 

Time to finish job is 3 minutes.

Run QE on One Compute Node 4 GPUs

We need to bind UCX/OMPI with script from give URL. One important component behind the script is Nvidia Multi-Process-Service or MPS.

NVIDIA MPS improves the parallel performance by allowing multiple process to instantaneously share GPU compute resources.

Nvidia Tranditional: Full process isolation

Figure: Nvidia Tranditional: Full process isolation

Nvidia MPS: Full process isolation

Figure: Nvidia MPS: Full process isolation

Source: https://www.olcf.ornl.gov/wp-content/uploads/2021/06/MPS_ORNL_20210817.pdf

1
2
3
4
5
6
$ cd ~/qe/ausurf
$ wget $ wget https://raw.githubusercontent.com/LStuber/binding/master/binder.sh 
$ module unload *
$ module load quantum_espresso/v7.0
$ mpirun -n 4 binder.sh pw.x -input ausurf.in -npool 4

Time to finish job is 2 minutes.

Run QE on Multi-Nodes ?

Later comming soon…

References:

  1. MaX school on Advanced Materials and Molecular Modelling with Quantum ESPRESSO,
This post is licensed under CC BY 4.0 by the author.

Simplify Exascale Workflows with Singularity Container and Environment Modules

Reproducible Slurm Notebook