From 01652acc52331c1e38c0b8f482d92ace75deb648 Mon Sep 17 00:00:00 2001
From: paugier <pierre.augier@univ-grenoble-alpes.fr>
Date: Fri, 17 May 2024 23:02:27 +0200
Subject: [PATCH] Add bench_mpi/README.md

---
 bench/bench_mpi/README.md | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
 create mode 100644 bench/bench_mpi/README.md

diff --git a/bench/bench_mpi/README.md b/bench/bench_mpi/README.md
new file mode 100644
index 00000000..74d90a42
--- /dev/null
+++ b/bench/bench_mpi/README.md
@@ -0,0 +1,37 @@
+# Benchmark MPI with and without infiniband network
+
+It appears that the value of the environment variable `OMPI_MCA_pml` influences how mpi4py is built.
+Therefore, to use the infiniband network, mpi4py has to be rebuilt with something like:
+
+```sh
+export OMPI_MCA_pml=ucx
+pip install mpi4py --force-reinstall --no-binary mpi4py
+```
+
+At LEGI, there is a module `env/ib4openmpi` that sets this variable.
+
+The script `submit_legi.py` can use 2 virtual environments that should be built with something like this.
+
+First, get an interactive session on cores with the infiniband network:
+
+```sh
+oarsub -I -l "{net='ib' and os='bullseye'}/core=2,walltime=0:10:0"
+```
+
+Then,
+
+```sh
+rm -rf venv_mpi_ib venv_mpi_noib
+# module with infiniband
+module purge
+/usr/bin/python3 -m venv venv_mpi_ib
+. venv_mpi_ib/bin/activate
+module load env/ib4openmpi
+pip install mpi4py --no-binary mpi4py
+deactivate
+# module without infiniband
+module purge
+/usr/bin/python3 -m venv venv_mpi_noib
+. venv_mpi_noib/bin/activate
+pip install mpi4py --no-binary mpi4py
+```