|
version 0.6.0
|
|
|
version 0.6.0
|
The following guidelines describe the different ways to run Notus.
Notus runs only through mpirun command, at least on 1 processor. The description of physical case to simulate is present in an _.nts_ file of one of the test_cases subdirectory that must given on the command-line. For instance, to run Notus on 8 processors and solve the Poiseuille flow, the command is:
mpirun -np 8 notus test_cases/verification/navier/poiseuille_2D.nts
Notus full help:
usage: notus [OPTIONS] [--] CASE_FILE
CASE_FILE Notus case file (*.nts)
OPTIONS
-D TYPE:IDENT=VALUE Define an identifier outside the Notus case file
TYPE: integer|double|boolean|string
IDENT: identifier name
VALUE: identifier value corresponding to the type
-h, --help Display help and exit
-t NUM, --thread NUM Use NUM OpenMP threads. Has no effect if Notus is compiled without OpenMP.
--no-banner Do not display Notus banner
--no-color Disable the use of ANSI color codes
--cpu-time-file FILE File where the CPU time is output
--validation-file FILE File where the validation is output
The script notus.py enables one to run Notus in different modes :
There is also a Database mode which permmit to manipulate the database used for the non-regressions modes. For further infomation, see Manipulate non-regression database
Here is the Notus help, which presents the different modes available:
usage: notus.py [-h] [--no-color] MODE ...
Tool to execute Notus for non-regression, scalability or convergence tests.
positional arguments:
MODE sub-command help
non-regression run Notus non-regression tests.
non-regression-scalability
run Notus non-regression-scalability tests.
database Edit database.
scalability run Notus scalability tests.
convergence run Notus convergence tests.
run run Notus for parametric analysis.
Each mode use the same submitting options, it is explained how to use them in Submitting options section.
Information about the options of these different modes can be found in the following paragraphs.
It is the mode to use if you want to execute Notus. It allows to make parametric analysis.
Here is Notus full help for this mode:
usage: notus.py run [-h] [-n PROCS_NUMBER] [-f] [-m MPI_COMMAND] [-o OUTPUT]
[-b {black,white}] [-P PBS] [-S SLURM] [--no-color] [-v]
NTS_FILES [NTS_FILES ...]
positional arguments:
NTS_FILES Notus input files (.nts)
optional arguments:
-h, --help Show this help message and exit
-n PROCS_NUMBER, --procs-number PROCS_NUMBER Number of processors used tu run Notus
-f, --full Enable full validation
-m MPI_COMMAND, --mpi-command MPI_COMMAND MPI command (default: mpirun)
-o OUTPUT, --output OUTPUT Output directory of log files
-b {black,white}, --background {black,white} Colors for display are chosen according to the color of the background (default: black)
-P PBS, --PBS PBS Launch validation using a batch system (PBS)
-S SLURM, --SLURM SLURM Launch validation using a batch system (SLURM)
--no-color Disable the use of ANSI color codes
-v, --verbose Diplay output
More information about [Submitting] options can be found at Submitting options.
It is possible to run all or a part of the non regression tests, composed with verification and validation test cases (Verification and Validation) as well as various tests, thanks to the notus.py script. It is useful to check Notus portal on a new computer or to validate new developments.
Notus full help in Non-regression mode:
Usage: notus.py non-regression [-h] [-r REGEXP] [-s] [-f] [-m MPI_COMMAND]
[-o OUTPUT] [-b {black,white}] [-P PBS] [-S SLURM]
[--no-color] [-v]
Optional arguments:
-h, --help show this help message and exit
-r REGEXP, --regexp REGEXP filter test cases using a regular expression
-s, --sequential enable sequential non-regression tests
-f, --full enable full non-regression tests
-m MPI_COMMAND, --mpi-command MPI_COMMAND MPI command (default: mpirun)
-o OUTPUT, --output OUTPUT output directory of log files
-b {black,white}, --background {black,white} colors for display are chosen according to the color of the background (default: black)
-P PBS, --PBS PBS launch non-regression using a batch system (PBS)
-S SLURM, --SLURM SLURM launch non-regression using a batch system (SLURM)
--no-color Disable the use of ANSI color codes
-v, --verbose Diplay output
Since the script detects automatically the test cases to use, if you want to test only some of them, you will have to use the -r argument to filter those cases.
The full non-regression is possible thanks to the syntax introduced in every NTS file that makes it possible to run the same test case several times (to rotate the direction of flow for instance). For further information about this, see Add a test case to Notus test case bank
The SLURM template is yet to be written, which means that the non-regression using SLURM batch system is not available for now.
Results can be compared to some obtained on different hardware and software architecture present in the test_cases/results directory.
It is possible to run all the test related to the non regression of scalability, and compare the result with an existing database. It is usefull to check non regression scalability to check the capacity of Notus after new developments.
Notus full help in Non-regression-scalability mode:
usage: notus.py non-regression-scalability [Filters] [Submitting] [Options] database-path
positional arguments:
database_path Path of the scalability database (default: Pas de db par défault).
Options:
-h, --help show this help message and exit
-v, --verbose Verbose mode.
--reload {STANDBY,SUBMITTED,PASS,FAIL,ERROR} [{STANDBY,SUBMITTED,PASS,FAIL,ERROR} ...]
Reload cases by their status.
--summary [{STANDBY,SUBMITTED,PASS,FAIL,ERROR} [{STANDBY,SUBMITTED,PASS,FAIL,ERROR} ...]]
Print a summary and exit.
-o DIRECTORY, --directory DIRECTORY
Output directory (default: non-regression-scalability).
Filters:
--file-filter FILE_FILTER
Filter file names using a regular expression.
--parameter-filter PARAMETER_FILTER
Filter files by their paramaters using a regular expression.
[tips on how to use it with maybe some exemples]
It is possible to run a spatial or temporal convergence study using notus.py. The time step or the spatial step will have a geometrical evolution. It will be needed to put N_CELLS_X, N_CELLS_Y, N_CELLS_Z to make the mesh variable. It will be needed to put TIME_STEP to make the time step variable. The result of the study will be save as a markdown table, or on a css format. To automatically calculate convergence metrics, it is necessary to specify which output values should be tracked (e.g., max, min, average speed) in the post-processing block of the .nts file using the diagonstic quantity and grid_convergence_quantities keywords.
Notus full help in Convergence mode:
usage: notus.py convergence [Options] [Submission Parameters] [Convergence Parameters] NTS_FILE [NTS_FILE ...]
positional arguments:
nts_files link to the .nts
Options:
-h, --help show this help message and exit
-v, --verbose Verbose mode.
--reload {STANDBY,SUBMITTED,PASS,FAIL,ERROR} [{STANDBY,SUBMITTED,PASS,FAIL,ERROR} ...]
Reload cases by their status.
-o DIRECTORY, --directory DIRECTORY
Output directory (default: convergence).
Convergence Parameters:
--type {space,time} choose between time and space
--time-step TIME_STEP
give the value of the time step, example 1.0, default value 1.0, if the problem has no dependency on time, the value does not matter
--final-time FINAL_TIME
indicates the time when the calcul ends (optional for spatial convergence)
--initial-mesh INITIAL_MESH [INITIAL_MESH ...]
spacial convergence : give the first mesh, the others will have a lower step-value following a geometric factor time convergence : the mesh used
--geometric-factor GEOMETRIC_FACTOR
geometric factor for the convergence so n step turn into n × geometric factors steps
--count COUNT number of simulations of the convergence study
Submission Parameters:
-n MPI_PROCS, --mpi-procs MPI_PROCS
Force the number of MPI processes.
--mpi-command MPI_COMMAND
MPI command (default: mpirun).
--dual-pass Submit in dual pass batch system.
--batch-template BATCH_TEMPLATE
Template file for batch jobs.
--batch-command BATCH_COMMAND
Command to launch batch jobs.
--slurm BATCH_TEMPLATE
Submit in dual pass with Slurm.
--stage {submit,post-process}
Set the stage of the dual-pass batch system.
Examples:
Launch a spacial convergence study:
./notus.py convergence --dual-pass --batch-template path/to/template/file --batch-command "sbatch {BATCH_FILE}" --stage submit --type space --initial-mesh 250 100 --geometric-factor 2 path/to/nts_files.nts
./notus.py convergence --dual-pass --batch-template path/to/template/file --batch-command "sbatch {BATCH_FILE}" --stage post-process --type space --initial-mesh 250 100 --geometric-factor 2 path/to/nts_files.nts
Lauch a time convergence study:
./notus.py convergence [Submitting] --type time ... --geometric-factor 2 path/to/nts_files.nts
An example of the spatial convergence study of a problem called my_nts_2D_space.nts is given below:
./notus.py convergence –type space –initial-mesh 16 16 –geometric-factor 2 –count 4 -n 12 -o /tmp/space_cv my_nts_2D_space.nts
Warning: the file (my_nts_2D_space.nts) should not define convergence study variables, e.g. N_CELLS_X, N_CELLS_Y in the case of 2D spatial convergence.
An example of the temporal convergence study of a problem called my_nts_2D_time.nts is given below:
./notus.py convergence –type time –time-step 1e-2 –initial-mesh 16 16 –geometric-factor 2 –count 4 -n 12 -o /tmp/time_cv my_nts_2D_time.nts
my_nts_2D_space.nts launch on a cluster using SLURM is given below:First, the submission with slurm
./notus.py convergence –slurm path_template_batch –stage submit –type space –initial-mesh 16 16 –geometric-factor 2 –count 4 -n 12 -o /tmp/space_cv my_nts_2D_space.nts
Then, the post-processing
./notus.py convergence –slurm path_template_batch –stage post-process –type space –initial-mesh 16 16 –geometric-factor 2 –count 4 -n 12 -o /tmp/space_cv my_nts_2D_space.nts
Warning: please give the path for the slurm template file path_template_batch.
To see a complete working example of spatial and temporal convergence study, using a bash script, please take a look at tools/run_convegence/space and tools/run_convegence/time.
It is possible to run different types of scalability study: weak, strong, strong node.
Notus full help in Scalability mode:
usage: notus.py scalability [-h] SUBMODE ...
positional arguments:
SUBMODE Scalability type
weak Run a weak scalability test on several nodes.
strong Run a strong scalability test on several nodes.
strong-node
Run a strong scalability test on one node.
options:
-h, --help show this help message and exit
The strong scalability evaluate the efficiency of workload reduction per core by increasing the number of cores while keeping the constant mesh size.
Notus full help in Scalability weak mode:
usage: notus.py scalability weak [Submitting] [Scalability parameters] [Options] nts_files [nts_files ...]
positional arguments:
nts_files nts files used to run scalability study
Options:
-h, --help show this help message and exit
-v, --verbose Verbose mode.
--reload {STANDBY,SUBMITTED,PASS,FAIL,ERROR} [{STANDBY,SUBMITTED,PASS,FAIL,ERROR} ...]
Reload cases by their status.
--output-format {ssv,csv}
Set the output format: Space or Comma Separated Value (default = ssv)
-o DIRECTORY, --directory DIRECTORY
Output directory (default: scalability_weak).
Submitting:
--mpi-command MPI_COMMAND
MPI command (default: mpirun).
--dual-pass Submit in dual pass batch system.
--batch-template BATCH_TEMPLATE
Template file for batch jobs.
--batch-command BATCH_COMMAND
Command to launch batch jobs.
--slurm BATCH_TEMPLATE
Submit in dual pass with Slurm.
--stage {submit,post-process}
Set the stage of the dual-pass batch system.
Scalability parameters:
--time-iterations TIME_ITERATIONS
Number of time iterations
--dimension {2,3} Dimension of the case (default = 3)
--arithmetic MIN COMMON-DIFF MAXTERMS
Arithmetic progression (N° of nodes):
MIN = Initial term,
COMMON-DIFF = Common difference of the arithmetic sequence,
MAXTERMS = Maximum number of terms to add.
--geometric MIN COMMON-RATIO MAXEXP
Geometric progression (N° of nodes):
MIN = Initial term,
COMMON-RATIO = Common ratio of the geometric sequence,
MAXEXP = Maximum exponent (exponent starts at 0).
--custom NODES [NODES ...]
Custom progression (N° of nodes): N1 N2 N3 ...
--threads THREADS Number of OpenMP threads (default: 1).
--mpi-procs-per-node MPI_PROCS_PER_NODE
Number of MPI processes per node.
--cells-per-direction N_CELLS_X N_CELLS_Y (N_CELLS_Z) [N_CELLS_X N_CELLS_Y (N_CELLS_Z) ...]
Number of cells per direction per core.
Examples:
Launch a weak scalability test on a 3D case using the dual-pass job manager (arithmetic progression (4, 8, 12, 16, 20 nodes), 16 MPI processes per node, 2 OpenMP threads, 300 time iterations , 30^3 cells per core):
./notus.py scalability weak --dual-pass --stage submit --batch-template path/to/template/file --batch-command "sbatch {BATCH_FILE}" --arithmetic 4 4 4 --mpi-procs-per-node 16 --threads 2 --cells-per-direction 30 30 30 --time-iterations 300 paths/to/nts_files
./notus.py scalability weak --dual-pass --stage post-process --batch-template path/to/template/file --batch-command "sbatch {BATCH_FILE}" --arithmetic 4 4 4 --mpi-procs-per-node 16 --threads 2 --cells-per-direction 30 30 30 --time-iterations 300 paths/to/nts_files
tools/scalability/run_weak_curtaThe weak scalability evaluate code performance while maintaining a constant workload by increasing both at the same time the problem size and the core count.
Notus full help in Scalability strong mode:
usage: notus.py scalability strong [Submitting] [Scalability parameters] [Options] nts_files [nts_files ...]
positional arguments:
nts_files nts files used to run scalability study
Options:
-h, --help show this help message and exit
-v, --verbose Verbose mode.
--reload {STANDBY,SUBMITTED,PASS,FAIL,ERROR} [{STANDBY,SUBMITTED,PASS,FAIL,ERROR} ...]
Reload cases by their status.
--output-format {ssv,csv}
Set the output format: Space or Comma Separated Value (default = ssv)
-o DIRECTORY, --directory DIRECTORY
Output directory (default: scalability_strong).
Submitting:
--mpi-command MPI_COMMAND
MPI command (default: mpirun).
--dual-pass Submit in dual pass batch system.
--batch-template BATCH_TEMPLATE
Template file for batch jobs.
--batch-command BATCH_COMMAND
Command to launch batch jobs.
--slurm BATCH_TEMPLATE
Submit in dual pass with Slurm.
--stage {submit,post-process}
Set the stage of the dual-pass batch system.
Scalability parameters:
--time-iterations TIME_ITERATIONS
Number of time iterations
--dimension {2,3} Dimension of the case (default = 3)
--arithmetic MIN COMMON-DIFF MAXTERMS
Arithmetic progression (N° of nodes):
MIN = Initial term,
COMMON-DIFF = Common difference of the arithmetic sequence,
MAXTERMS = Maximum number of terms to add.
--geometric MIN COMMON-RATIO MAXEXP
Geometric progression (N° of nodes):
MIN = Initial term,
COMMON-RATIO = Common ratio of the geometric sequence,
MAXEXP = Maximum exponent (exponent starts at 0).
--custom NODES [NODES ...]
Custom progression (N° of nodes): N1 N2 N3 ...
--threads THREADS Number of OpenMP threads (default: 1).
--mpi-procs-per-node MPI_PROCS_PER_NODE
Number of MPI processes per node.
--cells-per-direction N_CELLS_X N_CELLS_Y (N_CELLS_Z) [N_CELLS_X N_CELLS_Y (N_CELLS_Z) ...]
Number of cells per direction.
Examples:
Launch a strong scalability test on a 2D case using the dual-pass job manager (geometric progression (4, 8, 16, 32, 64 nodes), 32 MPI processes per node, 60 time iterations , 500^2 cells (total)):
./notus.py scalability strong --dual-pass --stage submit --batch-template path/to/template/file --batch-command "sbatch {BATCH_FILE}" --geometric 4 4 4 --mpi-procs-per-node 32 --dimension 2 --cells-per-direction 500 500 --time-iterations 60 paths/to/nts_files
./notus.py scalability strong --dual-pass --stage post-process --batch-template path/to/template/file --batch-command "sbatch {BATCH_FILE}" --geometric 4 4 4 --mpi-procs-per-node 32 --dimension 2 --cells-per-direction 500 500 500 --time-iterations 60 paths/to/nts_files
tools/scalability/run_strong_MPIopenMP for both irene-rome TGCC and curta MCIA cluster.For both strong and weak modes, it will be needed to put TIME_ITERATIONS in the nts file to control from the oustide of nts file the maximum number of iterations.
1.11.0