Home Quantum ESPRESSO on exascale.mahidol.ac.th Multi-Node-Multi-GPUs MNMG Testing
Post
Cancel

Quantum ESPRESSO on exascale.mahidol.ac.th Multi-Node-Multi-GPUs MNMG Testing

Quantum ESPRESSO is an integrated suite of Open-Source computer codes for electronic-structure calculations and materials modeling at the nanoscale based on density-functional theory, plane waves, and pseudopotentials.

What can PWscf do ?

PWscf performs many different kinds of self-consistent calculations of electronic-structure properties within Density-Functional Theory (DFT), using a Plane-Wave (PW) basis set and pseudopotentials (PP).[1]

In particular:

  • ground-state energy and one-electron (Kohn-Sham) orbitals, atomic forces, stresses;

  • structural optimization, also with variable cell;

  • molecular dynamics on the Born-Oppenheimer surface, also with variable cell;

  • macroscopic polarization (and orbital magnetization) via Berry Phases;

  • various forms of finite electric fields, with a sawtooth potential or with the modern theory of polarization;

  • Effective Screening Medium (ESM) method

Download Dataset:

The environment variable BENCHMARK_DIR will be used throughout the example to refer to the directory containing the AUSURF112 input files.

1
2
3
4
5
6
cd ~
mdkir qe
mkdir qe/ausurf
cd qe/ausurf
wget https://repository.prace-ri.eu/git/UEABS/ueabs/-/raw/master/quantum_espresso/test_cases/small/Au.pbe-nd-van.UPF
wget https://repository.prace-ri.eu/git/UEABS/ueabs/-/raw/master/quantum_espresso/test_cases/small/ausurf.in

Run QE on One Compute Node One GPU

As suggestion rules for runing on GPU QE on HPC and GPU systems:

  • 1 GPU : 1 MPI
  • Fill the CPU with OpenMP threads
  • No task groups (-ntg 1)
  • No parallel eigensolver (-ndiag 1), no diagonized?
1
2
3
4
5
$ cd ~/qe/ausurf

$ module unload *
$ module load quantum_espresso/v7.0
$ mpirun -n 1  pw.x -npool 1 -ntg 1 -ndiag 1 -input ausurf.in 

Time to finish job is 3 minutes.

Run QE on One Compute Node 4 GPUs

We need to bind UCX/OMPI with script from give URL. One important component behind the script is Nvidia Multi-Process-Service or MPS.

NVIDIA MPS improves the parallel performance by allowing multiple process to instantaneously share GPU compute resources.

Nvidia Tranditional: Full process isolation

Figure: Nvidia Tranditional: Full process isolation

Nvidia MPS: Full process isolation

Figure: Nvidia MPS: Full process isolation

Source: https://www.olcf.ornl.gov/wp-content/uploads/2021/06/MPS_ORNL_20210817.pdf

1
2
3
4
5
6
$ cd ~/qe/ausurf
$ wget $ wget https://raw.githubusercontent.com/LStuber/binding/master/binder.sh 
$ module unload *
$ module load quantum_espresso/v7.0
$ mpirun -n 4 binder.sh pw.x -input ausurf.in -npool 4

Time to finish job is 2 minutes.

Run QE on Multi-Nodes ?

Later comming soon…

References:

  1. MaX school on Advanced Materials and Molecular Modelling with Quantum ESPRESSO,
This post is licensed under CC BY 4.0 by the author.