MPI exchange

Communications to update overlapping field regions. More...

Namespaces
module	mod_mpi_exchange
	MPI Exchange modules between processors.
module	mod_prepare_mpi_exchange
	Prepare mpi_exchange.
module	mod_prepare_mpi_exchange_list
	MPI Exchange preparation for communication of node-list-like arrays.
module	type_mpi_exchange
	Data types for MPI Exchange communications.
module	variables_mpi_exchange
	Define variables associated to MPI communications on cells and faces.

Functions
subroutine	check_mpi_exchange_vector_cell ()
	Check mpi exchange data for a vector defined on cells.

Detailed Description

An MPI Exchange updates overlapping regions of a field.

  …      …

  ↕      ↕
┌──┐   ┌──┐
│p1│ ↔ │p8│ ↔  …
└──┘   └──┘
  ↕      ↕
┌──┐   ┌──┐
│p0│ ↔ │p7│ ↔  …
└──┘   └──┘

The domain attached to a processor contaings overlapping regions; each one corresponds to the physical domain of a neighbor process.

To exchange fields, process topology and sub-grid indexes must be gathered in a dedicated derived type instance: a derivative of type_mpi_exchange::t_mpi_exchange_base. To gather data do:

call prepare_mpi_exchange_cell(exchange_data)
call prepare_mpi_exchange_face_u(exchange_data)
call prepare_mpi_exchange_face_v(exchange_data)
call prepare_mpi_exchange_face_w(exchange_data)
call prepare_mpi_exchange_list(exchange_data, node_list, node_type)

according to the grid onto which your field is defined. Then, just do:

call mpi_exchange(field)

— Documentation in progress below —

Preparation steps

1. Evaluate the number and rank of neighbor processors.
2. Evaluate the size of communications.

Detail on communications

The MPI Exchange sub-domains are defined as follows for cell-centered grids:

┏━━━━━━┳━━━━━━━━━━━━━━     ━━━━━━━━━━━━━━┳━━━━━━┓
┃      ┃                                 ┃      ┃
┣━━━━━━╋━━━━━━┯━━━━━━━     ━━━━━━━┯━━━━━━╋━━━━━━┫
┃      ┃      │                   │      ┃      ┃
┃      ┠──────┼───────     ───────┼──────┨      ┃
┃      ┃      │                   │      ┃      ┃
┃o    o┃o    o│                   │o    o┃o    o┃
        ╵    ╵                     ╵    ╵
        is   is+(is-2)     ie-(is-2)   ie
                      SEND

┏━━━━━━┳━━━━━━━━━━━━━━     ━━━━━━━━━━━━━━┳━━━━━━┓
┃      ┃                                 ┃      ┃
┣━━━━━━╋━━━━━━┯━━━━━━━     ━━━━━━━┯━━━━━━╋━━━━━━┫
┃      ┃      │                   │      ┃      ┃
┃      ┠──────┼───────     ───────┼──────┨      ┃
┃      ┃      │                   │      ┃      ┃
┃o    o┃o    o│                   │o    o┃o    o┃
 ╵    ╵                                   ╵    ╵
 1   is-1                                ie+1 nx
                      RECV

The corners are sent multiple times.

The MPI Exchange sub-domains are defined as follows for face-centered grids:

┏━━━━━━┳━━━━━━━━━━━━━━     ━━━━━━━━━━━━━━┳━━━━━━┓
┃      ┃                                 ┃      ┃
┣━━━━━━╋━━━━━━┯━━━━━━━     ━━━━━━━┯━━━━━━╋━━━━━━┫
┃      ┃      │                   │      ┃      ┃
┃      ┠──────┼───────     ───────┼──────┨      ┃
┃      ┃      │                   │      ┃      ┃
┃ │  │ ┃ │  │ │                   │ │  │ ┃ │  │ ┃
       ╹    ╵                       ╵    ╹
      isu  isu+(isu-2)     ieu-(isu-2)  ieu
                      SEND

┏━━━━━━┳━━━━━━━━━━━━━━     ━━━━━━━━━━━━━━┳━━━━━━┓
┃      ┃                                 ┃      ┃
┣━━━━━━╋━━━━━━┯━━━━━━━     ━━━━━━━┯━━━━━━╋━━━━━━┫
┃      ┃      │                   │      ┃      ┃
┃      ┠──────┼───────     ───────┼──────┨      ┃
┃      ┃      │                   │      ┃      ┃
┃ │  │ ┃ │  │ │                   │ │  │ ┃ │  │ ┃
╹    ╵                                     ╵    ╹
1  isu-1                                 ieu+1  nxu
                      RECV