A guide to sending security advisory e-mails
============================================

--------------------------------------------------------
Step 1: Prepare the advisory texts, patches and metadata
--------------------------------------------------------

[details are covered elsewhere]

----------------------------------
Step 2: Prepare the website update
----------------------------------

  $ cd ${PMC_AREA_WC}/security
  $ ${TRUNK_WC}/tools/dist/advisory.py generate \
        --destination=${SITE_WC}/publish/security \
        CVE-2015-5259 CVE-2015-5343 ...

This will generate a plain-text version of the advisories, including
patches etc., suitable for publishing on our web site. Once these
are generated, make sure you add the links to the new files to:

    ${SITE_WC}/publish/security/index.html


-----------------------------------------------
Step 3: Check the advisories and their metadata
-----------------------------------------------

  $ cd ${PMC_AREA_WC}/security
  $ ${TRUNK_WC}/tools/dist/advisory.py test \
        --username=someone \
        --revision=22091347 \
        --release-versions=1.8.15,1.9.3 \
        --release-date=2015-12-15 \
        CVE-2015-5259 CVE-2015-5343 ...

Assuming all the required bits are in place, this will generate the
complete text of a GPG-signed e-mail message, signed by and sent from
someone@apache.org, for all the listed CVE numbers.

Note the arguments:

    --revision    is the revision on
                  https://dist.apache.org/repos/dist/dev/subversion
                  in which the tarballs are/will be available
                  (see: notice-template.txt in ${PMC_AREA_WC}/security).

    --release-versions   is a comma-separated list of version numbers
                         in which fixes for the CVE numbers will be
                         available.

    --release-date       is the expected date of the release(s).


----------------------
Step 4: Send the mails
----------------------

  $ cd ${PMC_AREA_WC}/security
  $ ${TRUNK_WC}/tools/dist/advisory.py send \
        (the rest of the arguments are as in step 3).

The mails will be sent one at a time to each recipient separately.


--------------------------------------------------
Step 5: Wait for the release. Release.
        Commit the site update prepared in step 1.
--------------------------------------------------



TODO: security/mailer.py does not calculate the micalg= PGP/MIME
      parameter based on the properties of the actual PGP key
      used. It's currently hard-coded as "pgp-sha512" which *should*
      be correct for anyone signing these mails with their ASF release
      signing key.

A guide to the various backport scripts:
========================================

There two primary functions:

F1. Auto-merge bot; the nightly svn-role commits.

F2. Conflicts detector bot; the svn-backport-conflicts-1.9.x buildbot task.

And two interactive functions¹:

F3. Reviewing STATUS nominations and casting votes.

F4. Adding new entries to STATUS.



The scripts are:

backport.pl:
    oldest script, implements [F1], [F2], and [F3].  As of Feb 2018, used in
    production by svn-role (running on svn-qavm3) and by svn-backport-conflicts-1.9.x
    (a buildbot job).

nominate.pl:
    Symlink to backport.pl.  Implements [F4].  (The script inspects its argv[0].)

backport_tests_pl.py:
    Regression tests for backport.pl.


backport/*.py:
    New Python-based library implementation of STATUS parsing (status.py) and
    of merging nominated revisions (merger.py).  Colloquially referred to as
    'backport.py', even though there is no script by that name.  Written in
    Python 3.

    The modules include their unit tests, see 'python3 -munittest
    backport.status' and 'python3 -munittest backport.merger'.  However,
    changes to these files should be tested both by the unit tests and by the
    backport_tests_py.py blackbox/regression tests.

detect-conflicting-backports.py:
    Implementation of [F2] using backport.py.
    Not currently used in production.

merge-approved-backports.py:
    Implementation of [F1] using backport.py.
    Not currently used in production.

backport_tests_py.py:
    Regression tests for detect-conflicting-backports.py and merge-approved-backports.py


backport_tests.py:
    Common part of backport_tests_pl.py and backport_tests_py.py.  Uses the
    svntest framework (../../subversion/tests/cmdline/svntest/), which is
    written in Python 2.

    Note that backport_tests.py and backport/*.py are written in different
    languages, so they never 'import' each other.  backport_tests.py invokes
    detect-conflicting-backports.py, merge-approved-backports.py, and
    backport.pl in the same manner: through subprocess.check_call().

backport_tests_data/backport*.dump:
    Expected output files for backport_tests.py; see the BackportTest
    decorator in backport_tests.py.


All scripts can be run with '--help' to display their usage messages.

backport.pl is considered deprecated since backport.py is better architected
and is written in a language that many more active developers are comfortable
with.  The unattended jobs [F1] and [F2] should be converted to using
backport.py whenever someone gets around to do the legwork.  The interactive
versions [F3] and [F4] are still in active use, however, so the physical
backport.pl script should be kept around until Python versions of these are
available.


TODO: document that "Notes: ... --accept=foo ..." is parsed, see backport_tests.py #3.


¹ For backport.pl's interactive features, see:
<http://mail-archives.apache.org/mod_mbox/subversion-dev/201503.mbox/%3c20150304225114.GD2036@tarsus.local2%3e>