README.md
1### `include/elemental/core/dist_matrix/`
2
3This folder contains the header files for the various partial specializations of
4the `DistMatrix` class; please see `src/core/dist_matrix` for the
5corresponding source files. Each specialization involves choosing a
6sensical pairing of distributions for the rows and columns of the matrix:
7
8- `CIRC`/"o": Only give the data to a single process
9- `STAR`/"\*": Give the data to every process
10- `MC`: Distribute round-robin within each column of the 2D process grid (*M*atrix *C*olumn)
11- `MR`: Distribute round-robin within each row of the 2D process grid (*M*atrix *R*ow)
12- `VC`: Distribute round-robin within a column-major ordering of the entire
13 2D process grid (*V*ector *C*olumn)
14- `VR`: Distribute round-robin within a row-major ordering of the entire
15 2D process grid (*V*ector *R*ow)
16- `MD`: Distribute round-robin over a diagonal of the tiling of the 2D process
17 grid (*M*atrix *D*iagonal)
18
19The valid pairings are:
20
21| Distribution | ColComm | RowComm | DistComm | RedundantComm | CrossComm |
22|:------------:|:-------:|:-------:|:---------:|:-------------:|:---------:|
23| `(o ,o )` | self | self | self | self | `VC` |
24| `(* ,* )` | self | self | self | `VC` | self |
25| `(MD,* )` | `MD` | self | `MD` | self | `MDPerp` |
26| `(* ,MD)` | self | `MD` | `MD` | self | `MDPerp` |
27| `(MC,MR)` | `MC` | `MR` | `VC` | self | self |
28| `(MR,MC)` | `MR` | `MC` | `VR` | self | self |
29| `(MC,* )` | `MC` | self | `MC` | `MR` | self |
30| `(* ,MC)` | self | `MC` | `MC` | `MR` | self |
31| `(MR,* )` | `MR` | self | `MR` | `MC` | self |
32| `(* ,MR)` | self | `MR` | `MR` | `MC` | self |
33| `(VC,* )` | `VC` | self | `VC` | self | self |
34| `(* ,VC)` | self | `VC` | `VC` | self | self |
35| `(VR,* )` | `VR` | self | `VR` | self | self |
36| `(* ,VR)` | self | `VR` | `VR` | self | self |
37
38where `DistComm` refers to the communicator that the entire matrix (rather than
39just the rows or columns) is distributed over. When the matrix is distributed
40over a communicator which only involves only a subset of the processes, it is
41possible to either assign the data to just that subset or redundantly store
42the entire matrix on each such subset of processes (e.g., within each row of a
432D arrangement of the set of processes). The `RedundantComm` refers to the
44communicator where each member process stores the same information, and the
45`CrossComm` is the communicator where only a single process (the *root*) is
46assigned any data.
47
48To make this discussion more precise, each valid matrix distribution for
49`DistMatrix` logically arranges the set of `p` processes of the `r` by `c`
50process grid into a 4D mesh: `ColComm` x `RowComm` x `RedundantComm` x `CrossComm`, where `DistComm` is equal to `ColComm` x `RowComm`.
51
52We are now ready to describe the contents of this folder (in addition to this
53file):
54
55- `Abstract.hpp`: The underlying distribution-agnostic base class
56- `CIRC_CIRC.hpp`: The `<T,CIRC,CIRC>` specialization, which provides a
57 distributed matrix where only one process owns data. It provides a simple
58 mechanism for forming a matrix on a single process and then redistributing
59 into another distribution, e.g., `(MC,MR)`.
60- `MC_MR.hpp`: The standard matrix distribution
61- `MC_STAR.hpp`: Only distribute each column like a standard matrix
62 distribution
63- `MD_STAR.hpp`: Distribute each column like the diagonal of the standard
64 matrix distribution
65- `MR_MC.hpp`: The transpose of the standard matrix distribution
66- `MR_STAR.hpp`: Distribute each column like the row of a standard matrix
67 distribution
68- `STAR_MC.hpp`: Distribute each row like a column of the standard matrix
69 distribution
70- `STAR_MD.hpp`: Distribute each row like the diagonal of a standard matrix
71 distribution
72- `STAR_MR.hpp`: Distribute each row like a standard matrix distribution
73- `STAR_STAR.hpp`: Give each process a full copy of the matrix
74- `STAR_VC.hpp`: Distribute each row using a round-robin wrapping over a
75 column-major ordering of the process grid
76- `STAR_VR.hpp`: Distribute each row using a round-robin wrapping over a
77 row-major ordering of the process grid
78- `VC_STAR.hpp`: Distribute each column using a round-robin wrapping over a
79 column-major ordering of the process grid
80- `VR_STAR.hpp`: Distribute each column using a round-robin wrapping over a
81 row-major ordering of the process grid
82