The following guidelines describe how to manipulate the database used by the non-regression and non-regression-scalability modes.

Introduction

The non-regression process works by comparing the values produced by a fresh run against reference values stored in a database. The database mode of notus.py is the tool used to build and maintain that database: adding new reference cases, updating existing ones, inspecting their content, removing them, and cleaning up entries that no longer match the test cases on disk.

This guide covers only how to manipulate the database. The non-regression and non-regression-scalability runs that produce and compare against these values are documented separately (see Non-regression testing with notus.py).

This page covers every database operation and the options they share.

The two databases

By default, databases live in test_cases/database/.

The non-regression database (test_cases/database/non-regression.db) is common to everyone and tracked with Git. It stores the reference values used to verify and validate the code.
A non-regression-scalability database is specific to each machine, because CPU times depend on the hardware. It is therefore created per computer and not shared.

A database is a JSON file. Each entry is keyed by an NTS file path (relative to test_cases/), then by a hash of the parameter set, and holds the reference values together with their tolerances.

Common options

All operations share the same options:

Option	Role	Default
`--database-path PATH`	Database file to operate on.	`test_cases/database/non-regression.db`
`--directory DIR`	Where to read run results from (for `--add` / `--replace`).	`non-regression`
`--default-tolerance T`	Absolute tolerance for standard values.	`1e-14`
`--default-convergence-tolerance T`	Tolerance for convergence cases.	`1e-2`
`--default-time-tolerance T`	Relative tolerance (percent) for CPU times.	`1%`
`--default-scalability-tolerance T`	Tolerance for scalability efficiency.	`1e-1`
`-v, --verbose`	Verbose output.	off

Note: Every operation takes a FILTER, which is a regular expression matched against the NTS file path — not an exact file name. A filter such as poiseuille_2D selects every case whose path contains it, so a single command can act on several cases at once. The leading test_cases/ prefix is optional: test_cases/verification/navier/poiseuille_2D.nts and verification/navier/poiseuille_2D.nts both match.

Add

After running Notus in non-regression (or non-regression-scalability) mode — see Non-regression testing with notus.py for how to launch those runs — add a new test case to the database from the results you just obtained:

  ./notus.py database --add poiseuille_2D

Since --database-path defaults to the non-regression database, the command above is equivalent to:

  ./notus.py database --add poiseuille_2D --database-path test_cases/database/non-regression.db

Choose the tolerance with the --default-* options. For example, to add the 2D Poiseuille flow with a 1e-11 tolerance:

  ./notus.py database --add poiseuille_2D --default-tolerance 1e-11

Warning: --add reads the run results from the directory given by --directory (default non-regression). After a non-regression-scalability run, the results are in non-regression-scalability, so remember to pass --directory non-regression-scalability when adding scalability cases.

Replace

--replace updates the reference values of cases that are already in the database. It behaves like --add, but skips any case that is not yet stored (it reports that there is nothing to replace and suggests --add instead).

  ./notus.py database --replace poiseuille_2D

Use --replace when you have re-run an existing case and want to refresh its stored values without risking the accidental creation of new entries. This keeps a clean separation between "introduce a new reference" (--add) and "update an existing reference" (--replace).

Edit

--edit changes values for cases that already exist, without needing to re-run Notus first. The script asks which parameter set to modify, then which value, and lets you change either the reference value or the tolerance:

  ./notus.py database --edit poiseuille_2D.nts

A typical interactive session:

  Reference values:

      Name      │          Value          │ Tolerance
  ──────────────┼─────────────────────────┼──────────
  mean velocity │ 1.00173015541651922e-01 │ 1e-13

  Edit this set of parameter? (N[o]|y[es]) yes
  Edit 'mean velocity'? (N[o]|y[es]) yes
  Change the reference value of 'mean velocity'? (N[o]|y[es]) no
  Change the tolerance of 'mean velocity'? (N[o]|y[es]) yes
  Enter the new tolerance of 'mean velocity': 1e-11
  ────────────────────────────────

To change the tolerance of every value at once, combine --edit with a --default-* option; you then select the file and parameter set to apply it to, and save your changes.

Show

--show prints what is currently stored, for the whole database or for a subset:

  ./notus.py database --show poiseuille_2D

  ./notus.py database --show poiseuille_2D --database-path path/to/database.db

For convergence and scalability cases, the values are printed as a table (mesh / error / order or node / time / efficiency), exactly as they appear during a run.

Remove

--remove deletes a case from the database:

  ./notus.py database --remove poiseuille_2D

You can remove a whole file, or only a specific parameter set of that file. The script walks you through the choice interactively:

  Match in database: 'verification/navier/poiseuille_2D_convergence.nts'
  Remove this file from the database? (N[o]|y[es]) no
  Remove a specific set of parameters for this file? (N[o]|y[es]) yes
  File: 'verification/navier/poiseuille_2D_convergence.nts'
  No parameters
  Convergence: space

  Mean velocity :

    Mesh   │     mean velocity     │ Tolerance │ Local order │ Tolerance
  ─────────┼───────────────────────┼───────────┼─────────────┼───────────
  (10, 10) │ 1.005000274462560e-01 │ 1.0       │ n/a         │ 0.1
  (20, 20) │ 1.001250029479938e-01 │ 1.0       │ n/a         │ 0.1
  (40, 40) │ 1.000312502359342e-01 │ 1.0       │ 2.000       │ 0.1

  Remove this convergence case? (N[o]|y[es]) yes
   ════════════════════════════════
  Save changes from the remove function? (N[o]|y[es]) yes
  Database updated

Clean

Over time the database can accumulate orphan entries: references that no longer correspond to anything on disk. --clean scans the database and reports two kinds of orphan:

Missing NTS files — the database references an NTS file that no longer exists on disk. The whole entry is removed.
Outdated parameter sets — the NTS file still exists, but a stored parameter set no longer matches its current @PARAMETER definitions (for example, a parameter value was removed from the file). Only the offending parameter set is removed.

Run it on the whole database, or restrict it to a subset with an optional filter:

  ./notus.py database --clean

  ./notus.py database --clean poiseuille

The script lists what it found, asks for confirmation, and only then deletes:

  Missing NTS files:
    'verification/navier/old_case.nts'
  Outdated parameter sets:
    'verification/navier/poiseuille_2D.nts'  →  [['integer', 'direction', '3']]

  Delete these orphan entries? (y[es]|N[o]) yes
  Orphan entries deleted.

If nothing is wrong, it reports Database is clean: nothing to remove. and stops.

Note: --clean handles convergence cases (whose parameter sets are computed per refinement level) but skips scalability cases for now.

Tolerance logic

The comparison rule differs between value types.

For standard non-regression values, the tolerance is absolute: \(|V_\text{db} - V_\text{run}| < \text{tolerance}\), typically between 1e-8 and 1e-14.
For scalability (CPU times), the tolerance is relative: \(|V_\text{db} - V_\text{run}| / V_\text{db} < \text{tolerance}\), typically a few percent, since timings vary across machines.

FAQ

What is the difference between –add and –replace?

--add introduces a new reference case (and can update an existing one). --replace only updates cases already present; it never creates a new entry. Use --replace to refresh values safely without the risk of adding something unintended.

My filter matches several cases. Is that normal?

Yes. The FILTER argument is a regular expression on the file path, so it intentionally matches every case whose path contains it. Use a more specific pattern (for instance the full relative path) to target a single case.

–add says it cannot find my scalability results.

--add reads from the directory given by --directory, which defaults to non-regression. Scalability results are written to non-regression-scalability, so add --directory non-regression-scalability.

What does –clean remove exactly?

Only entries that no longer correspond to the test cases on disk: references to deleted NTS files, and stored parameter sets that no longer match the file's current @PARAMETER definitions. It always asks for confirmation before deleting.