1 MPICH2 Release 1.5
2
3MPICH2 is a high-performance and widely portable implementation of the
4MPI-2.2 standard from the Argonne National Laboratory. This release
5has all MPI 2.2 functions and features required by the standard with
6the exception of support for the "external32" portable I/O format and
7user-defined data representations for I/O.
8
9This README file should contain enough information to get you started
10with MPICH2. More extensive installation and user guides can be found
11in the doc/installguide/install.pdf and doc/userguide/user.pdf files
12respectively. Additional information regarding the contents of the
13release can be found in the CHANGES file in the top-level directory,
14and in the RELEASE_NOTES file, where certain restrictions are
15detailed. Finally, the MPICH2 web site,
16http://www.mcs.anl.gov/research/projects/mpich2, contains information
17on bug fixes and new releases.
18
19
201. Getting Started
212. Reporting Installation or Usage Problems
223. Compiler Flags
234. Alternate Channels and Devices
245. Alternate Process Managers
256. Alternate Configure Options
267. Testing the MPICH2 installation
278. Fault Tolerance
289. Environment Variables
2910. Developer Builds
3011. Installing MPICH2 on windows
3112. Multiple Fortran compiler support
32
33
34-------------------------------------------------------------------------
35
361. Getting Started
37==================
38
39The following instructions take you through a sequence of steps to get
40the default configuration (ch3 device, nemesis channel (with TCP and
41shared memory), Hydra process management) of MPICH2 up and running.
42
43(a) You will need the following prerequisites.
44
45 - REQUIRED: This tar file mpich2-1.5.tar.gz
46
47 - REQUIRED: A C compiler (gcc is sufficient)
48
49 - OPTIONAL: A C++ compiler, if C++ applications are to be used
50 (g++, etc.). If you do not require support for C++ applications,
51 you can disable this support using the configure option
52 --disable-cxx (configuring MPICH2 is described in step 1(d)
53 below).
54
55 - OPTIONAL: A Fortran 77 compiler, if Fortran 77 applications are
56 to be used (gfortran, ifort, etc.). If you do not require
57 support for Fortran 77 applications, you can disable this
58 support using --disable-f77 (configuring MPICH2 is described in
59 step 1(d) below).
60
61 - OPTIONAL: A Fortran 90 compiler, if Fortran 90 applications are
62 to be used (gfortran, ifort, etc.). If you do not require
63 support for Fortran 90 applications, you can disable this
64 support using --disable-fc. Note that Fortran 77 support is a
65 prerequisite for Fortran 90 support (configuring MPICH2 is
66 described in step 1(d) below).
67
68 Also, you need to know what shell you are using since different shell
69 has different command syntax. Command "echo $SHELL" prints out the
70 current shell used by your terminal program.
71
72(b) Unpack the tar file and go to the top level directory:
73
74 tar xzf mpich2-1.5.tar.gz
75 cd mpich2-1.5
76
77 If your tar doesn't accept the z option, use
78
79 gunzip mpich2-1.5.tar.gz
80 tar xf mpich2-1.5.tar
81 cd mpich2-1.5
82
83(c) Choose an installation directory, say
84 /home/<USERNAME>/mpich2-install, which is assumed to non-existent
85 or empty. It will be most convenient if this directory is shared
86 by all of the machines where you intend to run processes. If not,
87 you will have to duplicate it on the other machines after
88 installation.
89
90(d) Configure MPICH2 specifying the installation directory:
91
92 for csh and tcsh:
93
94 ./configure --prefix=/home/<USERNAME>/mpich2-install |& tee c.txt
95
96 for bash and sh:
97
98 ./configure --prefix=/home/<USERNAME>/mpich2-install 2>&1 | tee c.txt
99
100 Bourne-like shells, sh and bash, accept "2>&1 |". Csh-like shell,
101 csh and tcsh, accept "|&". If a failure occurs, the configure
102 command will display the error. Most errors are straight-forward
103 to follow. For example, if the configure command fails with:
104
105 "No Fortran 77 compiler found. If you don't need to build any
106 Fortran programs, you can disable Fortran support using
107 --disable-f77 and --disable-fc. If you do want to build
108 Fortran programs, you need to install a Fortran compiler such
109 as gfortran or ifort before you can proceed."
110
111 ... it means that you don't have a Fortran compiler :-). You will
112 need to either install one, or disable Fortran support in MPICH2.
113
114 If you are unable to understand what went wrong, please go to step
115 (2) below, for reporting the issue to the MPICH2 developers and
116 other users.
117
118(e) Build MPICH2:
119
120 for csh and tcsh:
121
122 make |& tee m.txt
123
124 for bash and sh:
125
126 make 2>&1 | tee m.txt
127
128 This step should succeed if there were no problems with the
129 preceding step. Check file m.txt. If there were problems, do a
130 "make clean" and then run make again with V=1.
131
132 make V=1 |& tee m.txt (for csh and tcsh)
133
134 OR
135
136 make V=1 2>&1 | tee m.txt (for bash and sh)
137
138 Then go to step (2) below, for reporting the issue to the MPICH2
139 developers and other users.
140
141(f) Install the MPICH2 commands:
142
143 for csh and tcsh:
144
145 make install |& tee mi.txt
146
147 for bash and sh:
148
149 make install 2>&1 | tee mi.txt
150
151 This step collects all required executables and scripts in the bin
152 subdirectory of the directory specified by the prefix argument to
153 configure.
154
155(g) Add the bin subdirectory of the installation directory to your
156 path in your startup script (.bashrc for bash, .cshrc for csh,
157 etc.):
158
159 for csh and tcsh:
160
161 setenv PATH /home/<USERNAME>/mpich2-install/bin:$PATH
162
163 for bash and sh:
164
165 PATH=/home/<USERNAME>/mpich2-install/bin:$PATH ; export PATH
166
167 Check that everything is in order at this point by doing:
168
169 which mpicc
170 which mpiexec
171
172 These commands should display the path to your bin subdirectory of
173 your install directory.
174
175 IMPORTANT NOTE: The install directory has to be visible at exactly
176 the same path on all machines you want to run your applications
177 on. This is typically achieved by installing MPICH2 on a shared
178 NFS file-system. If you do not have a shared NFS directory, you
179 will need to manually copy the install directory to all machines
180 at exactly the same location.
181
182(h) MPICH2 uses a process manager for starting MPI applications. The
183 process manager provides the "mpiexec" executable, together with
184 other utility executables. MPICH2 comes packaged with multiple
185 process managers; the default is called Hydra.
186
187 Now we will run an MPI job, using the mpiexec command as specified
188 in the MPI-2 standard. There are some examples in the install
189 directory, which you have already put in your path, as well as in
190 the directory mpich2-1.5/examples. One of them is the
191 classic CPI example, which computes the value of pi by numerical
192 integration in parallel.
193
194 To run the CPI example with 'n' processes on your local machine,
195 you can use:
196
197 mpiexec -n <number> ./examples/cpi
198
199 Test that you can run an 'n' process CPI job on multiple nodes:
200
201 mpiexec -f machinefile -n <number> ./examples/cpi
202
203 The 'machinefile' is of the form:
204
205 host1
206 host2:2
207 host3:4 # Random comments
208 host4:1
209
210 'host1', 'host2', 'host3' and 'host4' are the hostnames of the
211 machines you want to run the job on. The ':2', ':4', ':1' segments
212 depict the number of processes you want to run on each node. If
213 nothing is specified, ':1' is assumed.
214
215 More details on interacting with Hydra can be found at
216 http://wiki.mcs.anl.gov/mpich2/index.php/Using_the_Hydra_Process_Manager
217
218If you have completed all of the above steps, you have successfully
219installed MPICH2 and run an MPI example.
220
221-------------------------------------------------------------------------
222
2232. Reporting Installation or Usage Problems
224===========================================
225
226[VERY IMPORTANT: PLEASE COMPRESS ALL FILES BEFORE SENDING THEM TO
227US. DO NOT SPAM THE MAILING LIST WITH LARGE ATTACHMENTS.]
228
229The distribution has been tested by us on a variety of machines in our
230environments as well as our partner institutes. If you have problems
231with the installation or usage of MPICH2, please follow these steps:
232
2331. First see the Frequently Asked Questions (FAQ) page at
234http://wiki.mcs.anl.gov/mpich2/index.php/Frequently_Asked_Questions to
235see if the problem you are facing has a simple solution. Many common
236problems and their solutions are listed here.
237
2382. If you cannot find an answer on the FAQ page, look through previous
239email threads on the mpich-discuss mailing list archive
240(http://lists.mcs.anl.gov/pipermail/mpich-discuss/). It is likely
241someone else had a similar problem, which has already been resolved
242before.
243
2443. If neither of the above steps work, please send an email to
245mpich-discuss@mcs.anl.gov. You need to subscribe to this list
246(https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss) before
247sending an email.
248
249Your email should contain the following files. ONCE AGAIN, PLEASE
250COMPRESS BEFORE SENDING, AS THE FILES CAN BE LARGE. Note that,
251depending on which step the build failed, some of the files might not
252exist.
253
254 mpich2-1.5/c.txt (generated in step 1(d) above)
255 mpich2-1.5/m.txt (generated in step 1(e) above)
256 mpich2-1.5/mi.txt (generated in step 1(f) above)
257 mpich2-1.5/config.log (generated in step 1(d) above)
258 mpich2-1.5/src/openpa/config.log (generated in step 1(d) above)
259 mpich2-1.5/src/mpl/config.log (generated in step 1(d) above)
260 mpich2-1.5/src/pm/hydra/config.log (generated in step 1(d) above)
261 mpich2-1.5/src/pm/hydra/tools/topo/hwloc/hwloc/config.log (generated in step 1(d) above)
262
263 DID WE MENTION? DO NOT FORGET TO COMPRESS THESE FILES!
264
265If you have compiled MPICH2 and are having trouble running an
266application, please provide the output of the following command in
267your email.
268
269 mpiexec -info
270
271Finally, please include the actual error you are seeing when running
272the application, including the mpiexec command used, and the host
273file. If possible, please try to reproduce the error with a smaller
274application or benchmark and send that along in your bug report.
275
2764. If you have found a bug in MPICH2, we request that you report it at
277our bug tracking system:
278(https://trac.mcs.anl.gov/projects/mpich2/newticket). Even if you
279believe you have found a bug, we recommend you sending an email to
280mpich-discuss@mcs.anl.gov first.
281
282
283-------------------------------------------------------------------------
284
2853. Compiler Flags
286=================
287
288MPICH2 allows several sets of compiler flags to be used. The first
289three sets are configure-time options for MPICH2, while the fourth is
290only relevant when compiling applications with mpicc and friends.
291
292(a) CFLAGS, CPPFLAGS, CXXFLAGS, FFLAGS, FCFLAGS, LDFLAGS and LIBS
293(abbreviated as xFLAGS): Setting these flags would result in the
294MPICH2 library being compiled/linked with these flags and the flags
295internally being used in mpicc and friends.
296
297(b) MPICH2LIB_CFLAGS, MPICH2LIB_CPPFLAGS, MPICH2LIB_CXXFLAGS,
298MPICH2LIB_FFLAGS, MPICH2LIB_FCFLAGS, MPICH2LIB_LDFLAGS and
299MPICH2LIB_LIBS (abbreviated as MPICH2LIB_xFLAGS): Setting these flags
300would result in the MPICH2 library being compiled/linked with these
301flags. However, these flags will *not* be used by mpicc and friends.
302
303(c) MPICH2_MAKE_CFLAGS: Setting these flags would result in MPICH2's
304configure tests to not use these flags, but the makefile's to use
305them. This is a temporary hack for certain cases that advanced
306developers might be interested in, but which break existing configure
307tests (e.g., -Werror). These are NOT recommended for regular users.
308
309(d) MPICH2_MPICC_FLAGS, MPICH2_MPICPP_FLAGS, MPICH2_MPICXX_FLAGS,
310MPICH2_MPIF77_FLAGS, MPICH2_MPIFC_FLAGS, MPICH2_LDFLAGS and
311MPICH2_LIBS (abbreviated as MPICH2_MPIX_FLAGS): These flags do *not*
312affect the compilation of the MPICH2 library itself, but will be
313internally used by mpicc and friends.
314
315
316 +--------------------------------------------------------------------+
317 | | | |
318 | | MPICH2 library | mpicc and friends |
319 | | | |
320 +--------------------+----------------------+------------------------+
321 | | | |
322 | xFLAGS | Yes | Yes |
323 | | | |
324 +--------------------+----------------------+------------------------+
325 | | | |
326 | MPICH2LIB_xFLAGS | Yes | No |
327 | | | |
328 +--------------------+----------------------+------------------------+
329 | | | |
330 | MPICH2_MAKE_xFLAGS | Yes | No |
331 | | | |
332 +--------------------+----------------------+------------------------+
333 | | | |
334 | MPICH2_MPIX_FLAGS | No | Yes |
335 | | | |
336 +--------------------+----------------------+------------------------+
337
338
339All these flags can be set as part of configure command or through
340environment variables.
341
342
343Default flags
344--------------
345By default, MPICH2 automatically adds certain compiler optimizations
346to MPICH2LIB_CFLAGS. The currently used optimization level is -O2.
347
348** IMPORTANT NOTE: Remember that this only affects the compilation of
349the MPICH2 library and is not used in the wrappers (mpicc and friends)
350that are used to compile your applications or other libraries.
351
352This optimization level can be changed with the --enable-fast option
353passed to configure. For example, to build an MPICH2 environment with
354-O3 for all language bindings, one can simply do:
355
356 ./configure --enable-fast=O3
357
358Or to disable all compiler optimizations, one can do:
359
360 ./configure --disable-fast
361
362For more details of --enable-fast, see the output of "configure
363--help".
364
365
366Examples
367--------
368
369Example 1:
370
371 ./configure --disable-fast MPICH2LIB_CFLAGS=-O3 MPICH2LIB_FFLAGS=-O3 \
372 MPICH2LIB_CXXFLAGS=-O3 MPICH2LIB_FCFLAGS=-O3
373
374This will cause the MPICH2 libraries to be built with -O3, and -O3
375will *not* be included in the mpicc and other MPI wrapper script.
376
377Example 2:
378
379 ./configure --disable-fast CFLAGS=-O3 FFLAGS=-O3 CXXFLAGS=-O3 FCFLAGS=-O3
380
381This will cause the MPICH2 libraries to be built with -O3, and -O3
382will be included in the mpicc and other MPI wrapper script.
383
384Example 3:
385
386There are certain compiler flags that should not be used with MPICH2's
387configure, e.g. gcc's -Werror, which would confuse configure and cause
388certain configure tests to fail to detect the correct system features.
389To use -Werror in building MPICH2 libraries, you can pass the compiler
390flags during the make step through the Makefile variable
391MPICH2_MAKE_CFLAGS as follows:
392
393 make MPICH2_MAKE_CFLAGS="-Wall -Werror"
394
395The content of MPICH2_MAKE_CFLAGS is appended to the CFLAGS in all
396relevant Makefiles.
397
398-------------------------------------------------------------------------
399
4004. Alternate Channels and Devices
401=================================
402
403The communication mechanisms in MPICH2 are called "devices". MPICH2
404supports ch3 (default), as well as many third-party devices that are
405released and maintained by other institutes such as osu_ch3 (from Ohio
406State University for InfiniBand and iWARP), ch_mx (from Myricom for
407Myrinet MX), etc.
408
409 *************************************
410
411ch3 device
412**********
413The ch3 device contains different internal communication options
414called "channels". We currently support nemesis (default) and sock
415channels.
416
417nemesis channel
418---------------
419Nemesis provides communication using different networks (tcp, mx) as
420well as various shared-memory optimizations. To configure MPICH2 with
421nemesis, you can use the following configure option:
422
423 --with-device=ch3:nemesis
424
425The TCP network module gets configured in by default. To specify a
426different network module such as MX, you can use:
427
428 --with-device=ch3:nemesis:mx
429
430If the MX include files and libraries are not in the normal search
431paths, you can specify them with the following options:
432
433 --with-mx-include= and --with-mx-lib=
434
435... or the if lib/ and include/ are in the same directory, you can use
436the following option:
437
438 --with-mx=
439
440If the MX libraries are shared libraries, they need to be in the
441shared library search path. This can be done by adding the path to
442/etc/ld.so.conf, or by setting the LD_LIBRARY_PATH variable in your
443.bashrc (or .tcshrc) file. It's also possible to set the shared
444library search path in the binary. If you're using gcc, you can do
445this by adding
446
447 LD_LIBRARY_PATH=/path/to/lib
448
449 (and)
450
451 LDFLAGS="-Wl,-rpath -Wl,/path/to/lib"
452
453... as arguments to configure.
454
455By default, MX allows for only eight endpoints per node causing
456ch3:nemesis:mx to give initialization errors with greater than 8
457processes on the same node (this is an MX error and not an inherent
458limitation in the MPICH2/Nemesis design). If needed, this can be set
459to a higher number when MX is loaded. We recommend the user to contact
460help@myri.com for details on how to do this.
461
462Shared-memory optimizations are enabled by default to improve
463performance for multi-processor/multi-core platforms. They can be
464disabled (at the cost of performance) either by setting the
465environment variable MPICH_NO_LOCAL to 1, or using the following
466configure option:
467
468 --enable-nemesis-dbg-nolocal
469
470The --with-shared-memory= configure option allows you to choose how
471Nemesis allocates shared memory. The options are "auto", "sysv", and
472"mmap". Using "sysv" will allocate shared memory using the System V
473shmget(), shmat(), etc. functions. Using "mmap" will allocate shared
474memory by creating a file (in /dev/shm if it exists, otherwise /tmp),
475then mmap() the file. The default is "auto". Note that System V
476shared memory has limits on the size of shared memory segments so
477using this for Nemesis may limit the number of processes that can be
478started on a single node.
479
480sock channel
481------------
482sock is the traditional TCP sockets based communication channel. It
483uses TCP/IP sockets for all communication including intra-node
484communication. So, though the performance of this channel is worse
485than that of nemesis, it should work on almost every platform. This
486channel can be configured using the following option:
487
488 --with-device=ch3:sock
489
490
491pamid device
492************
493This is the device used on the IBM Blue Gene/Q system. The following
494configure options can be used:
495
496 ./configure --host=powerpc64-bgq-linux --target=powerpc64-bgq-linux --build=powerpc64-linux-gnu
497 --with-device=pamid:BGQ --with-file-system=bg+bglockless --enable-timer-type=device
498 --with-fwrapname=fmpich --with-cross=src/mpid/pamid/cross/bgq8
499 --with-pm=none --with-mpe=no --enable-timing=no --disable-collchk
500 --disable-graphics --disable-rlog --disable-sample --disable-rpath
501 --with-aint-size=8 --with-assert-level=2 --enable-fast=O3
502 --enable-error-messages --disable-debuginfo
503 --enable-thread-cs=per-object --enable-handle-allocation=tls
504 --enable-refcount=lock-free --disable-predefined-refcount
505 CC=/bgsys/drivers/V1R1M2/ppc64/gnu-linux/bin/powerpc64-bgq-linux-gcc
506 CXX=/bgsys/drivers/V1R1M2/ppc64/gnu-linux/bin/powerpc64-bgq-linux-g++
507 F77=/bgsys/drivers/V1R1M2/ppc64/gnu-linux/bin/powerpc64-bgq-linux-gfortran
508 FC=/bgsys/drivers/V1R1M2/ppc64/gnu-linux/bin/powerpc64-bgq-linux-gfortran
509
510-------------------------------------------------------------------------
511
5125. Alternate Process Managers
513=============================
514
515hydra
516-----
517Hydra is the default process management framework that uses existing
518daemons on nodes (e.g., ssh, pbs, slurm, sge) to start MPI
519processes. More information on Hydra can be found at
520http://wiki.mcs.anl.gov/mpich2/index.php/Using_the_Hydra_Process_Manager
521
522mpd
523---
524MPD was the traditional process manager in MPICH2. The file
525mpich2-1.5/src/pm/mpd/README has more information about
526interactive commands for managing the ring of MPDs. The MPD process
527manager is now deprecated.
528
529smpd
530----
531SMPD is a process manager for interoperability between Microsoft
532Windows and UNIX, where some processes are running on Windows and
533others are running on a variant of UNIX. For more information, please
534see mpich2-1.5/src/pm/smpd/README.
535
536gforker
537-------
538gforker is a process manager that creates processes on a single
539machine, by having mpiexec directly fork and exec them. gforker is
540mostly meant as a research platform and for debugging purposes, as it
541is only meant for single-node systems.
542
543slurm
544-----
545SLURM is an external process manager not distributed with
546MPICH2. MPICH2's default process manager, hydra, has native support
547for slurm and you can directly use it in slurm environments (it will
548automatically detect slurm and use slurm capabilities). However, if
549you want to use the slurm provided "srun" process manager, you can use
550the "--with-pmi=slurm --with-pm=no" option with configure. Note that
551the "srun" process manager that comes with slurm uses an older PMI
552standard which does not have some of the performance enhancements that
553hydra provides in slurm environments.
554
555-------------------------------------------------------------------------
556
5576. Alternate Configure Options
558==============================
559
560MPICH2 has a number of other features. If you are exploring MPICH2 as
561part of a development project, you might want to tweak the MPICH2
562build with the following configure options. A complete list of
563configuration options can be found using:
564
565 ./configure --help
566
567However, for your convenience, we list a few important options here:
568
569Performance Options:
570
571 --enable-fast - Turns off error checking and collection of internal
572 timing information
573
574 --enable-timing=no - Turns off just the collection of internal timing
575 information
576
577 --enable-ndebug - Turns on NDEBUG, which disables asserts. This is a
578 subset of the optimizations provided by
579 enable-fast, but is useful in environments where
580 the user wishes to retain the debug symbols, e.g.,
581 this can be combined with the --enable-g option.
582
583MPI Features:
584
585 --enable-romio - Build the ROMIO implementation of MPI-IO (enabled
586 by default).
587
588 --with-file-system - When used with --enable-romio, specifies
589 filesystems ROMIO should support. They can be
590 specified by passing them in a '+'-delimited
591 list: (e.g.,
592 --with-file-system="pvfs+nfs+ufs").
593
594 If you have installed version 2 of the PVFS
595 file system, you can use the
596 '--with-pvfs2=<prefix>' configure option to
597 specify where libraries, headers, and utilities
598 have been installed. If you have added the pvfs
599 utilities to your PATH, then ROMIO will detect
600 this and build support for PVFS automatically.
601
602 --enable-threads - Build MPICH2 with support for multi-threaded
603 applications. Only the sock and nemesis channels
604 support MPI_THREAD_MULTIPLE.
605
606 --with-thread-package - When used with --enable-threads, this option
607 specifies the thread package to use. This
608 option defaults to "posix". At the moment,
609 only POSIX threads are supported on UNIX
610 platforms. We plan to support Solaris
611 threads in the future.
612
613Language bindings:
614
615 --enable-f77 - Build the Fortran 77 bindings (enabled by default).
616
617 --enable-fc - Build the Fortran 90 bindings (enabled by default).
618
619 --enable-cxx - Build the C++ bindings (enabled by default).
620
621Shared library support:
622
623 --enable-shared - Enable shared library support. Note that "rpath"
624 linking is used by default in stock MPICH2
625 distributions. Re-running ./autogen.sh with a
626 patched libtool can eliminate rpath usage. See
627 http://wiki.debian.org/RpathIssue for more info.
628
629Cross compilation:
630
631 --with-cross=filename - Provide values for the tests that required
632 running a program, such as the tests that
633 configure uses to determine the sizes of the
634 basic types. This should be a fine in
635 Bourne shell format containing variable
636 assignment of the form
637
638 CROSS_SIZEOF_INT=2
639
640 for all of the CROSS_xxx variables.
641
642Error checking and reporting:
643
644 --enable-error-checking=level - Control the amount of error
645 checking. Currently, only "no" and
646 "all" is supported; all is the
647 default.
648
649 --enable-error-messages=level - Control the aount of detail in error
650 messages. By default, MPICH2
651 provides instance-specific error
652 messages; but, with this option,
653 MPICH2 can be configured to provide
654 less detailed messages. This may be
655 desirable on small systems, such as
656 clusters built from game consoles or
657 high-density massively parallel
658 systems. This is still under active
659 development.
660
661Compilation options for development:
662
663 --enable-g=value - Controls the amount of debugging information
664 collected by the code. The most useful choice
665 here is dbg, which compiles with -g.
666
667 --enable-coverage - An experimental option that enables GNU coverage
668 analysis.
669
670 --with-logging=name - Select a logging library for recording the
671 timings of the internal routines. We have
672 used this to understand the performance of the
673 internals of MPICH2. More information on the
674 logging options, capabilities and usage can be
675 found in doc/logging/logging.pdf.
676
677 --enable-timer-type=name - Select the timer to use for MPI_Wtime and
678 internal timestamps. name may be one of:
679 gethrtime - Solaris timer (Solaris systems
680 only)
681 clock_gettime - Posix timer (where available)
682 gettimeofday - Most Unix systems
683 linux86_cycle - Linux x86; returns cycle
684 counts, not time in seconds*
685 linuxalpha_cycle - Like linux86_cycle, but for
686 Linux Alpha*
687 gcc_ia64_cycle - IPF ar.itc timer*
688 device - The timer is provided by the device
689 *Note that the cycle timers are intended to be used by
690 MPICH2 developers for internal low-level timing.
691 Normal users should not use these as they are not
692 guaranteed to be accurate in certain situations.
693
694-------------------------------------------------------------------------
695
6967. Testing the MPICH2 installation
697==================================
698
699To test MPICH2, we package the MPICH2 test suite in the MPICH2
700distribution. You can run the test suite using:
701
702 make testing
703
704The results summary will be placed in test/summary.xml
705
706-------------------------------------------------------------------------
707
7088. Fault Tolerance
709==================
710
711MPICH2 has some tolerance to process failures, and supports
712checkpointing and restart.
713
714Tolerance to Process Failures
715-----------------------------
716
717The features described in this section should be considered
718experimental. Which means that they have not been fully tested, and
719the behavior may change in future releases. The below notes are some
720guidelines on what can be expected in this feature:
721
722 - ERROR RETURNS: Communication failures in MPICH2 are not fatal
723 errors. This means that if the user sets the error handler to
724 MPI_ERRORS_RETURN, MPICH2 will return an appropriate error code in
725 the event of a communication failure. When a process detects a
726 failure when communicating with another process, it will consider
727 the other process as having failed and will no longer attempt to
728 communicate with that process. The user can, however, continue
729 making communication calls to other processes. Any outstanding
730 send or receive operations to a failed process, or wildcard
731 receives (i.e., with MPI_ANY_SOURCE) posted to communicators with a
732 failed process, will be immediately completed with an appropriate
733 error code.
734
735 - COLLECTIVES: For collective operations performed on communicators
736 with a failed process, the collective would return an error on
737 some, but not necessarily all processes. A collective call
738 returning MPI_SUCCESS on a given process means that the part of the
739 collective performed by that process has been successful.
740
741 - PROCESS MANAGER: If used with the hydra process manager, hydra will
742 detect failed processes and notify the MPICH2 library. Users can
743 query the list of failed processes using MPIX_Comm_group_failed().
744 This functions returns a group consisting of the failed processes
745 in the communicator. The function MPIX_Comm_remote_group_failed()
746 is provided for querying failed processes in the remote processes
747 of an intercommunicator.
748
749 Note that hydra by default will abort the entire application when
750 any process terminates before calling MPI_Finalize. In order to
751 allow an application to continue running despite failed processes,
752 you will need to pass the -disable-auto-cleanup option to mpiexec.
753
754 - FAILURE NOTIFICATION: THIS IS AN UNSUPPORTED FEATURE AND WILL
755 ALMOST CERTAINLY CHANGE IN THE FUTURE!
756
757 In the current release, hydra notifies the MPICH2 library of failed
758 processes by sending a SIGUSR1 signal. The application can catch
759 this signal to be notified of failed processes. If the application
760 replaces the library's signal handler with its own, the application
761 must be sure to call the library's handler from it's own
762 handler. Note that you cannot call any MPI function from inside a
763 signal handler.
764
765Checkpoint and Restart
766----------------------
767
768MPICH2 supports checkpointing and restart fault-tolerance using BLCR.
769
770CONFIGURATION
771
772First, you need to have BLCR version 0.8.2 or later installed on your
773machine. If it's installed in the default system location, you don't
774need to do anything.
775
776If BLCR is not installed in the default system location, you'll need
777to tell MPICH2's configure where to find it. You might also need to
778set the LD_LIBRARY_PATH environment variable so that BLCR's shared
779libraries can be found. In this case add the following options to
780your configure command:
781
782 --with-blcr=<BLCR_INSTALL_DIR>
783 LD_LIBRARY_PATH=<BLCR_INSTALL_DIR>/lib
784
785where <BLCR_INSTALL_DIR> is the directory where BLCR has been
786installed (whatever was specified in --prefix when BLCR was
787configured).
788
789After it's configured compile as usual (e.g., make; make install).
790
791Note, checkpointing is only supported with the Hydra process manager.
792
793
794VERIFYING CHECKPOINTING SUPPORT
795
796Make sure MPICH2 is correctly configured with BLCR. You can do this
797using:
798
799 mpiexec -info
800
801This should display 'BLCR' under 'Checkpointing libraries available'.
802
803
804CHECKPOINTING THE APPLICATION
805
806There are two ways to cause the application to checkpoint. You can ask
807mpiexec to periodically checkpoint the application using the mpiexec
808option -ckpoint-interval (seconds):
809
810 mpiexec -ckpointlib blcr -ckpoint-prefix /tmp/app.ckpoint \
811 -ckpoint-interval 3600 -f hosts -n 4 ./app
812
813Alternatively, you can also manually force checkpointing by sending a
814SIGUSR1 signal to mpiexec.
815
816The checkpoint/restart parameters can also be controlled with the
817environment variables HYDRA_CKPOINTLIB, HYDRA_CKPOINT_PREFIX and
818HYDRA_CKPOINT_INTERVAL.
819
820To restart a process:
821
822 mpiexec -ckpointlib blcr -ckpoint-prefix /tmp/app.ckpoint -f hosts -n 4 -ckpoint-num <N>
823
824where <N> is the checkpoint number you want to restart from.
825
826These instructions can also be found on the MPICH2 wiki:
827
828 http://wiki.mcs.anl.gov/mpich2/index.php/Checkpointing
829
830-------------------------------------------------------------------------
831
8329. Environment Variables
833========================
834
835MPICH2 provides several environment variables that have different
836purposes. Many of these environment variables are listed in the
837README.envvar file. Some more are listed below.
838
839 MPICH_ASYNC_PROGRESS - Initiates a spare thread to provide
840 asynchronous progress. This improves progress semantics for
841 all MPI operations including point-to-point, collective,
842 one-sided operations and I/O. Setting this variable would
843 increase the thread-safety level to
844 MPI_THREAD_MULTIPLE. While this improves the progress
845 semantics, it might cause a small amount of performance
846 overhead for regular MPI operations.
847
848 ************************************
849
850 MPICH_NAMEPUB_DIR - Allows the user to override where the publish
851 and lookup information is placed for connect/accept based
852 applications.
853
854-------------------------------------------------------------------------
855
85610. Developer Builds
857====================
858For MPICH2 developers who want to directly work on the svn, there are
859a few additional steps involved (people using the release tarballs do
860not have to follow these steps). Details about these steps can be
861found here:
862http://wiki.mcs.anl.gov/mpich2/index.php/Getting_And_Building_MPICH2
863
864-------------------------------------------------------------------------
865
86611. Installing MPICH2 on Windows
867================================
868
869Here are the instructions for setting up MPICH2 on a Windows machine:
870
871(a) Install:
872 Microsoft Developer Studio 2003 or later
873 Intel Fortran 8.0 or later
874 cygwin
875 choose the dos file format option
876 install perl and svn
877
878(b) Checkout mpich2:
879
880 Bring up a command prompt.
881 (replace "yourname" with your MCS login name):
882 svn co https://svn.mcs.anl.gov/repos/mpi/mpich2/trunk mpich2
883
884(c) Generate *.h.in
885
886 Bring up a cygwin bash shell.
887 cd mpich2
888 ./autogen.sh
889 exit
890
891(d) Execute winconfigure.wsf
892
893(e) Open Developer Studio
894
895 open mpich2\mpich2.sln
896 build the ch3sockDebug mpich2 solution
897 build the ch3sockDebug mpich2s project
898 build the ch3sockRelease mpich2 solution
899 build the ch3sockRelease mpich2s project
900 build the Debug mpich2 solution
901 build the Release mpich2 solution
902 build the fortDebug mpich2 solution
903 build the fortRelease mpich2 solution
904 build the gfortDebug mpich2 solution
905 build the gfortRelease mpich2 solution
906 build the sfortDebug mpich2 solution
907 build the sfortRelease mpich2 solution
908
909(f) Open a command prompt
910
911 cd to mpich2\maint
912 execute "makegcclibs.bat"
913
914(g) Open another Developer Studio instance
915
916 open mpich2\examples\examples.sln
917 build the Release target of the cpi project
918
919(h) Return to Developer Studio with the mpich2 solution
920
921 set the version numbers in the Installer project
922 build the Installer mpich2 solution
923
924(i) Test and distribute mpich2\maint\ReleaseMSI\mpich2.msi
925
926 mpich2.msi can be renamed, eg mpich2-1.1.msi
927
928(j) To install the launcher:
929
930 Copy smpd.exe to a local directory on all the nodes.
931 Log on to each node as an administrator and execute "smpd.exe -install"
932
933(k) Compile and run an MPI application:
934
935 Compile an mpi application. Use mpi.h from mpich2\src\include\win32
936 and mpi.lib in mpich2\lib
937
938 Place your executable along with the mpich2 dlls somewhere accessable
939 to all the machines.
940
941 Execute a job by running something like: mpiexec -n 3 myapp.exe
942
943-------------------------------------------------------------------------
944
94512. Multiple Fortran compiler support
946=====================================
947
948If the C compiler that is used to build MPICH2 libraries supports both
949multiple weak symbols and multiple aliases of common symbols, the
950Fortran 77 binding can support multiple Fortran compilers. The
951multiple weak symbols support allow MPICH2 to provide different name
952mangling scheme (of subroutine names) required by differen Fortran
953compilers. The multiple aliases of common symbols support enables
954MPICH2 to equal different common block symbols of the MPI Fortran
955constant, e.g. MPI_IN_PLACE, MPI_STATUS_IGNORE. So they are understood
956by different Fortran compilers.
957
958Since the support of multiple aliases of common symbols is
959new/experimental, users can disable the feature by using configure
960option --disable-multi-aliases if it causes any undesirable effect,
961e.g. linker warnings of different sizes of common symbols, MPIFCMB*
962(the warning should be harmless).
963
964We have only tested this support on a limited set of
965platforms/compilers. On linux, if the C compiler that builds MPICH2
966is either gcc or icc, the above support will be enabled by configure.
967At the time of this writing, pgcc does not seem to have this multiple
968aliases of common symbols, so configure will detect the deficiency and
969disable the feature automatically. The tested Fortran compiler
970includes GNU Forran compilers(gfortan, g77), Intel Fortran
971compiler(ifort), Portland Group Fortran compilers(pgf77, pgf90),
972Absoft Fortran compilers (af77, af90), and IBM XL fortran
973compiler(xlf). What this mean is that if mpich2 is built by
974gcc/gfortran, the resulting mpich2 library can be used to link a
975Fortran program compiled/linked by another fortran compiler, say
976pgf77, say through mpif77 -f77=pgf77. As long as the Fortran program
977is linked without any errors by one of these compilers, the program
978shall be running fine.
979
1(C) 2011 by Argonne National Laboratory.
2 See COPYRIGHT in top-level directory.
3
4This file lists the various environment variables available to change the
5behavior of the MPICH2 library. These are intended to be used by advanced
6users.
7---------------------------------------------------------------------------
8
9MPICH_ALLTOALL_SHORT_MSG_SIZE
10 Aliases: MPIR_PARAM_ALLTOALL_SHORT_MSG_SIZE
11 Description: the short message algorithm will be used if the
12 per-destination message size (sendcount*size(sendtype)) is <= this
13 value
14 Default: 256
15
16MPICH_ALLTOALL_MEDIUM_MSG_SIZE
17 Aliases: MPIR_PARAM_ALLTOALL_MEDIUM_MSG_SIZE
18 Description: the medium message algorithm will be used if the
19 per-destination message size (sendcount*size(sendtype)) is <= this
20 value and larger than ALLTOALL_SHORT_MSG_SIZE
21 Default: 32768
22
23MPICH_ALLTOALL_THROTTLE
24 Aliases: MPIR_PARAM_ALLTOALL_THROTTLE
25 Description: max no. of irecvs/isends posted at a time in some alltoall
26 algorithms. Setting it to 0 causes all irecvs/isends to be posted
27 at once.
28 Default: 4
29
30MPICH_REDSCAT_COMMUTATIVE_LONG_MSG_SIZE
31 Aliases: MPIR_PARAM_REDSCAT_COMMUTATIVE_LONG_MSG_SIZE
32 Description: the long message algorithm will be used if the operation
33 is commutative and the send buffer size is >= this value (in bytes)
34 Default: 524288
35
36MPICH_BCAST_MIN_PROCS
37 Aliases: MPIR_PARAM_BCAST_MIN_PROCS
38 Description: the minimum number of processes in a communicator to use a
39 non-binomial broadcast algorithm
40 Default: 8
41
42MPICH_BCAST_SHORT_MSG_SIZE
43 Aliases: MPIR_PARAM_BCAST_SHORT_MSG_SIZE
44 Description: the short message algorithm will be used if the send
45 buffer size is < this value (in bytes)
46 Default: 12288
47
48MPICH_BCAST_LONG_MSG_SIZE
49 Aliases: MPIR_PARAM_BCAST_LONG_MSG_SIZE
50 Description: the long message algorithm will be used if the send buffer
51 size is >= this value (in bytes)
52 Default: 524288
53
54MPICH_ALLGATHER_SHORT_MSG_SIZE
55 Aliases: MPIR_PARAM_ALLGATHER_SHORT_MSG_SIZE
56 Description: For MPI_Allgather and MPI_Allgatherv, the short message
57 algorithm will be used if the send buffer size is < this value (in
58 bytes).
59 Default: 81920
60
61MPICH_ALLGATHER_LONG_MSG_SIZE
62 Aliases: MPIR_PARAM_ALLGATHER_LONG_MSG_SIZE
63 Description: For MPI_Allgather and MPI_Allgatherv, the long message
64 algorithm will be used if the send buffer size is >= this value (in
65 bytes)
66 Default: 524288
67
68MPICH_REDUCE_SHORT_MSG_SIZE
69 Aliases: MPIR_PARAM_REDUCE_SHORT_MSG_SIZE
70 Description: the short message algorithm will be used if the send
71 buffer size is <= this value (in bytes)
72 Default: 2048
73
74MPICH_ALLREDUCE_SHORT_MSG_SIZE
75 Aliases: MPIR_PARAM_ALLREDUCE_SHORT_MSG_SIZE
76 Description: the short message algorithm will be used if the send
77 buffer size is <= this value (in bytes)
78 Default: 2048
79
80MPICH_GATHER_VSMALL_MSG_SIZE
81 Aliases: MPIR_PARAM_GATHER_VSMALL_MSG_SIZE
82 Description: use a temporary buffer for intracommunicator MPI_Gather if
83 the send buffer size is < this value (in bytes)
84 Default: 1024
85
86MPICH_GATHER_INTER_SHORT_MSG_SIZE
87 Aliases: MPIR_PARAM_GATHER_INTER_SHORT_MSG_SIZE
88 Description: use the short message algorithm for intercommunicator
89 MPI_Gather if the send buffer size is < this value (in bytes)
90 Default: 2048
91
92MPICH_GATHERV_INTER_SSEND_MIN_PROCS
93 Aliases: MPIR_PARAM_GATHERV_INTER_SSEND_MIN_PROCS
94 MPICH2_GATHERV_MIN_PROCS
95 Description: Use Ssend (synchronous send) for intercommunicator
96 MPI_Gatherv if the "group B" size is >= this value. Specifying
97 "-1" always avoids using Ssend. For backwards compatibility,
98 specifying "0" uses the default value.
99 Default: 32
100
101MPICH_SCATTER_INTER_SHORT_MSG_SIZE
102 Aliases: MPIR_PARAM_SCATTER_INTER_SHORT_MSG_SIZE
103 Description: use the short message algorithm for intercommunicator
104 MPI_Scatter if the send buffer size is < this value (in bytes)
105 Default: 2048
106
107MPICH_ALLGATHERV_PIPELINE_MSG_SIZE
108 Aliases: MPIR_PARAM_ALLGATHERV_PIPELINE_MSG_SIZE
109 Description: The smallest message size that will be used for the
110 pipelined, large-message, ring algorithm in the MPI_Allgatherv
111 implementation.
112 Default: 32768
113
114MPICH_COMM_SPLIT_USE_QSORT
115 Aliases: MPIR_PARAM_COMM_SPLIT_USE_QSORT
116 Description: Use qsort(3) in the implementation of MPI_Comm_split
117 instead of bubble sort.
118 Default: 1
119
120MPICH_RMA_ACC_IMMED
121 Aliases: MPIR_PARAM_RMA_ACC_IMMED
122 Description: Use the immediate accumulate optimization
123 Default: 1
124
125MPICH_RMA_NREQUEST_THRESHOLD
126 Aliases: MPIR_PARAM_RMA_NREQUEST_THRESHOLD
127 Description: Threshold at which the RMA implementation attempts to
128 complete requests while completing RMA operations and while using
129 the lazy synchonization approach. Change this value if programs
130 fail because they run out of requests or other internal resources
131 Default: 4000
132
133MPICH_RMA_NREQUEST_NEW_THRESHOLD
134 Aliases: MPIR_PARAM_RMA_NREQUEST_NEW_THRESHOLD
135 Description: Threshold for the number of new requests since the last
136 attempt to complete pending requests. Higher values can increase
137 performance, but may run the risk of exceeding the available
138 number of requests or other internal resources.
139 Default: 128
140
141MPICH_NOLOCAL
142 Aliases: MPICH_NO_LOCAL
143 MPIR_PARAM_NOLOCAL
144 MPIR_PARAM_NO_LOCAL
145 Description: If true, force all processes to operate as though all
146 processes are located on another node. For example, this disables
147 shared memory communication hierarchical collectives.
148 Default: 0
149
150MPICH_ODD_EVEN_CLIQUES
151 Aliases: MPICH_EVEN_ODD_CLIQUES
152 MPIR_PARAM_ODD_EVEN_CLIQUES
153 MPIR_PARAM_EVEN_ODD_CLIQUES
154 Description: If true, odd procs on a node are seen as local to each
155 other, and even procs on a node are seen as local to each other.
156 Used for debugging on a single machine.
157 Default: 0
158
159MPICH_POLLS_BEFORE_YIELD
160 Aliases: MPIR_PARAM_POLLS_BEFORE_YIELD
161 Description: When MPICH is in a busy waiting loop, it will periodically
162 call a function to yield the processor. This parameter sets the
163 number of loops before the yield function is called. A value of 0
164 disables yielding.
165 Default: 1000
166
167MPICH_MEMDUMP
168 Aliases: MPIR_PARAM_MEMDUMP
169 Description: If true, list any memory that was allocated by MPICH2 and
170 that remains allocated when MPI_Finalize completes.
171 Default: 1
172
173MPICH_PROCTABLE_SIZE
174 Aliases: MPIR_PARAM_PROCTABLE_SIZE
175 Description: Size of the "MPIR" debugger interface proctable (process
176 table).
177 Default: 64
178
179MPICH_PROCTABLE_PRINT
180 Aliases: MPIR_PARAM_PROCTABLE_PRINT
181 Description: If true, dump the proctable entries at
182 MPIR_WaitForDebugger-time. (currently compile-time disabled by "#if
183 0")
184 Default: 1
185
186MPICH_ERROR_CHECKING
187 Aliases: MPIR_PARAM_ERROR_CHECKING
188 Description: If true, perform checks for errors, typically to verify
189 valid inputs to MPI routines. Only effective when MPICH2 is
190 configured with --enable-error-checking=runtime .
191 Default: 1
192
193MPICH_PRINT_ERROR_STACK
194 Aliases: MPIR_PARAM_PRINT_ERROR_STACK
195 Description: If true, print an error stack trace at error handling
196 time.
197 Default: 1
198
199MPICH_CHOP_ERROR_STACK
200 Aliases: MPIR_PARAM_CHOP_ERROR_STACK
201 Description: If >0, truncate error stack output lines this many
202 characters wide. If 0, do not truncate, and if <0 use a sensible
203 default.
204 Default: 0
205
206MPICH_NEM_LMT_DMA_THRESHOLD
207 Aliases: MPIR_PARAM_NEM_LMT_DMA_THRESHOLD
208 Description: Messages larger than this size will use the "dma" (knem)
209 intranode LMT implementation, if it is enabled and available.
210 Default: 2097152
211
212MPICH_NEMESIS_NETMOD
213 Aliases: MPIR_PARAM_NEMESIS_NETMOD
214 Description: If non-empty, this parameter specifies which network
215 module should be used for communication.
216 Default: ""
217
218MPICH_INTERFACE_HOSTNAME
219 Aliases: MPIR_PARAM_INTERFACE_HOSTNAME
220 Description: If non-NULL, this parameter specifies the IP address that
221 other processes should use when connecting to this process. This
222 parameter is mutually exclusive with the MPICH_NETWORK_IFACE
223 parameter and it is an error to set them both.
224 Default: NULL
225
226MPICH_NETWORK_IFACE
227 Aliases: MPIR_PARAM_NETWORK_IFACE
228 Description: If non-NULL, this parameter specifies which
229 pseudo-ethernet interface the tcp netmod should use (e.g., "eth1",
230 "ib0"). Note, this is a Linux-specific parameter. This parameter
231 is mutually exclusive with the MPICH_INTERFACE_HOSTNAME parameter
232 and it is an error to set them both.
233 Default: NULL
234
235MPICH_HOST_LOOKUP_RETRIES
236 Aliases: MPIR_PARAM_HOST_LOOKUP_RETRIES
237 Description: This parameter controls the number of times to retry the
238 gethostbyname() function before giving up.
239 Default: 10
240
241MPICH_SHM_EAGER_MAX_SZ
242 Aliases: MPIR_PARAM_SHM_EAGER_MAX_SZ
243 Description: This parameter controls the message size at which Nemesis
244 switches from eager to rendezvous mode. If this parameter is set
245 to -1, then Nemesis will choose an appropriate value.
246 Default: -1
247
248MPICH_SHM_READY_EAGER_MAX_SZ
249 Aliases: MPIR_PARAM_SHM_READY_EAGER_MAX_SZ
250 Description: This parameter controls the message size at which Nemesis
251 switches from eager to rendezvous mode for ready-send messages. If
252 this parameter is set to -1, then ready messages will always be
253 sent eagerly. If this parameter is set to -2, then Nemesis will
254 choose an appropriate value.
255 Default: -2
256
257MPICH_COMM_OVERRIDES
258 Aliases: MPIR_PARAM_COMM_OVERRIDES
259 Description: If set to false, communication override functionality will
260 be disabled for netmods that provide the override feature. Some
261 netmods may not honor this flag.
262 Default: 1
263
264MPICH_DEBUG_HOLD
265 Aliases: MPIR_PARAM_DEBUG_HOLD
266 Description: If true, causes processes to wait in MPI_Init and
267 MPI_Initthread for a debugger to be attached. Once the debugger
268 has attached, the variable 'hold' should be set to 0 in order to
269 allow the process to continue (e.g., in gdb, "set hold=0").
270 Default: 0
271
272MPICH_ENABLE_CKPOINT
273 Aliases: MPIR_PARAM_ENABLE_CKPOINT
274 Description: If true, enables checkpointing support and returns an
275 error if checkpointing library cannot be initialized.
276 Default: 0
277
278MPICH_ENABLE_COLL_FT_RET
279 Aliases: MPIR_PARAM_ENABLE_COLL_FT_RET
280 Description: NOT COMPATIBLE WITH NONBLOCKING COLLECTIVES (see
281 tt#1601)!!! Collectives called on a communicator with a failed
282 process should not hang, however the result of the operation may be
283 invalid even though the function returns MPI_SUCCESS. This option
284 enables an experimental feature that will return an error if the
285 result of the collective is invalid.
286 Default: 0
287
288MPICH_ABORT_ON_LEAKED_HANDLES
289 Aliases: MPIR_PARAM_ABORT_ON_LEAKED_HANDLES
290 Description: If true, MPI will call MPI_Abort at MPI_Finalize if any
291 MPI object handles have been leaked. For example, if MPI_Comm_dup
292 is called without calling a corresponding MPI_Comm_free. For
293 uninteresting reasons, enabling this option may prevent all known
294 object leaks from being reported. MPICH2 must have been configure
295 with "--enable-g=handlealloc" or better in order for this
296 functionality to work.
297 Default: 0
298
299MPICH_PORT_RANGE
300 Aliases: MPICH_PORTRANGE
301 MPIR_PARAM_PORT_RANGE
302 MPIR_PARAM_PORTRANGE
303 Description: The MPICH_PORT_RANGE environment variable allows you to
304 specify the range of TCP ports to be used by the process manager
305 and the MPICH2 library. The format of this variable is
306 <low>:<high>.
307 Default: {0,0}
308
309MPICH_CTXID_EAGER_SIZE
310 Aliases: MPIR_PARAM_CTXID_EAGER_SIZE
311 Description: The MPICH_CTXID_EAGER_SIZE environment variable allows you
312 to specify how many words in the context ID mask will be set aside
313 for the eager allocation protocol. If the application is running
314 out of context IDs, reducing this value may help.
315 Default: 2
316
317---------------------------------------------------------------------------
318
319Automatically generated
320 by: ./maint/genparams
321 at: Mon Oct 8 14:01:08 2012
322 from: src/util/param/params.yml (md5sum dabefd4ebb90a37693fa984f8332424f)
323
324