The following guidelines will help you to use Notus on various clusters.
- Note
- The choice of the allocated ressources for a given simulation (the number of processors asked) must be chosen carefully. Please follow the recommendations of 60,000 cells per core in 3D and 30,000 cells per core in 2D to be in the range of maximum efficiency of a parallel Notus run in full-MPI mode.
The usage of notus is described for the following computers:
- curta MCIA cluster website
- irene rome TGCC cluster website
Ssh configuration
SSH (Secure Shell) is critical for secure remote access. A well-configured SSH setup not only enhances security by using key-based authentication but also improves workflow efficiency by simplifying host aliases and file transfers.
Generating an SSH Key
Using a strong key (RSA 4096-bit) is recommended to ensure robust encryption.
ssh-keygen -t rsa -b 4096 -f mykey_clustername
Configuring SSH
Creating a configuration file **~/.ssh/config** reduces the need for long command-line options. Configuration could be:
Host curta
HostName curta2.mcia.fr
User username
PreferredAuthentications publickey
IdentityFile ~/.ssh/mykey_clustername
Host visu01
HostName visu01.mcia.fr
User username
ProxyJump myproxyserver
ForwardX11 yes
Host rome
HostName irene-amd-fr.ccc.cea.fr
User username
ProxyJump myproxyserver
ForwardX11 yes
ForwardX11Trusted yes
Advanced Tip of TGCC remote access: Consider using ProxyJump keyword if you need to go through a specific host to reach more secure networks.
Copying Your Public Key
For passwordless authentication, copy your public key to the remote machine:
ssh-copy-id -i mykey username@curta2.mcia.fr
File Transfer Using scp
With the configuration :
scp curta:/scratch/username/notus_run/myfolder /home/username/destination
Without configuration:
scp username@curta2.mcia.fr:/scratch/username/notus_run/myfolder /home/username/destination
Submission scripts
SLURM is a scalable workload manager designed for clusters and HPC environments. It efficiently allocates resources and schedules jobs to maximize cluster utilization, across nodes of a computer.
You need to write a SLURM file to have access to ressources
- For curta MCIA cluster URL
#!/bin/bash
#SBATCH --job-name=jobname
#SBATCH --time=06:00:00
#SBATCH --chdir=.
#SBATCH --output=%J_nameofoutputfile.out # %J is job id
#SBATCH --error=%J_nameoferrorfile.err # %J is job id
#SBATCH --partition=i2m
#SBATCH --nodes=8 # Nbr of node
#SBATCH --tasks-per-node=32 # tasks per node
# #SBATCH --tasks=32 # Nbr of tasks. It is another way of allocate ressources
set -e
NOTUS_PATH="the/path/of/your/binary/notus/file"
# Load required modules
source /etc/profile.d/modules.sh
module purge
module load notus_third_party/0.6.0/gcc/11.2.0
# Define parameters
myvar=1600
# Display job information
echo "#############################"
echo "User: $USER"
echo "Date: $(date)"
echo "Host: $(hostname)"
echo "Directory: $(pwd)"
echo "SLURM_JOBID: $SLURM_JOBID"
echo "SLURM_SUBMIT_DIR: $SLURM_SUBMIT_DIR"
echo "SLURM_JOB_NODELIST: $SLURM_JOB_NODELIST"
echo "#############################"
ulimit -s unlimited
mpirun -np $SLURM_NTASKS ${NOTUS_PATH} -Ddouble:my_variable=$myvar --no-color -- your_nts_file.nts
sync
exit 0
Submit a Job:
sbatch submission-curta.sh
Check Job Queue :
Cancel a Job :
For more advanced management, consult the SLURM documentation website which provides in-depth guidance on job dependencies, advanced scheduling options, and tuning.
- For irene rome TGCC cluster URL
Template for job Submission my_submission.template
:
#!/bin/bash
#MSUB -r XXXXX # Job name
#MSUB -T X # Elapsed time (s)
#MSUB -A gen00000 # alloc nbr
#MSUB -o PATH/log.X # Standard output
#MSUB -e PATH/log.X # Error output
#MSUB -q rome # partition
#MSUB -n X # Reservation of MPI processes
#MSUB -m scratch # location
PATH_NOTUS_BIN="$HOME/notus_path/notus"
PATH_RUN=`pwd`
set -x
ulimit -s unlimited
module purge
module load datadir/gen00000 python3/3.10.6 mkl/default mpi/openmpi
cd $BRIDGE_MSUB_PWD
ccc_mprun $PATH_NOTUS_BIN --no-color $PATH_RUN/mynts.nts
exit 0
Submit a Job:
ccc_msub my_submission.template
Check Job Queue :
Cancel a Job :