Building Releases
=================

Roundup is a pure Python application with no binary components. This file
describes how to build a source release. To find out how to install
Roundup, read the doc/installation.txt file.

Roundup release checklist:

1.  Regenerate locale/roundup.pot.  See "Extracting Translatable
    Messages" in doc/developers.txt.  This is best done some time in
    advance of the release to allow for translators to update their
    translations. Merge into .po files by running 'make *.po'
    in the locale directory. Run:

       cd locale
       make merge
       cd ..

    'make merge' will remake template if the source and html files are
    out of date. 'make template' will regenerate roundup.pot. Touch
    any .py file in the roundup/ directory to force 'make template'
    to run.

2.  Run unit tests! They should pass successfully. Install pytest
    using pip2/pip3 for python2 and python3. Then invoke pytest
    using both python versions from the top of the roundup tree:

        python3 -m pytest test/
        python2 -m pytest test/

3.  Update version in:
      CHANGES.txt (set date for version as well) 
      roundup/__init__.py
      website/www/index.txt
      website/www/conf.py  (also update copyright)
3a. Update license end date in COPYING.txt
3b. Update doc/acknowledgements.txt (add section for
    release, churn contributers etc.). (Use hg churn -c -r ####..####)

4.  Update documentation
      doc/announcement.txt
      doc/upgrading.txt
4a. cd to website/www and run 'make linkcheck'. See output in
    _tmp/linkcheck/output.txt
       fix broken references in docs
       verify redirects are correct

5.  Update setup.py info if needed (contacts, classifiers, etc.). When
    releasing check that Development Status matches release: stable,
    beta alpha etc.

    Check that metadata is valid and long descriptions is proper reST:

      python3 setup.py check --restructuredtext --metadata --strict
   
6.  Clean out all *.orig, *.rej, .#* files from the source.

      find . -name '*.orig' -exec rm {} \;
      find . -name '*.rej' -exec rm {} \;
      find . -name '.#*' -exec rm {} \;

6a. Rebuild .mo translation files in distribution

      cd locale
      make
      cd ..

7.  Remove previous build files

      python3 setup.py clean --all
      rm -rf build/share  # deletes locale .mo files

    Build including new .mo files built in 6a.

      python3 setup.py build

    (sdist generation will fail if this isn't done)

8.  Rebuild documentation in "share/doc/roundup/html"

      python3 setup.py build_doc

9.  Generate source distribution:

      python3 setup.py sdist

    (if you find sdist a little verbose, add "--quiet" to the end of the
     command)
9a. 2021/04/17 skip this for now. Need to make sure that whl installs
    executable scripts properly and update these directions to test.

       python2 setup.py bdist_wheel; python3 setup.py bdist_wheel

    to create binary distributions in wheel format. (egg format is
    deprecated.)

10. Check the roundup.egg-info/SOURCES.txt to make sure that any new files are
    included.  (use hg status --rev <last release or tag>:tip to list changed
    added and removed files. Last release e.g. 1.5.1 where tip is what would
    become 1.6) E.G.
      
      hg status --rev 2.0.0:tip | sed -ne 's/^A //p' | while read i ; \
      do echo $i; grep "$i" roundup.egg-info/SOURCES.txt; done | \
      uniq -c | sort -rn

    Anything with a count of 1 is not in the manifest.
    If there are missing files that should be in the manifest,
    edit MANIFEST.in to include them. For format docs see
    https://packaging.python.org/guides/using-manifest-in/#using-manifest-in
    (Note: files under website/ shouldn't be in the manifest.)
10a: Check for removed files still in manifest:

      hg status --rev 2.0.0:tip | sed -ne 's/^R //p' | while read i ; \
      do echo $i; grep "$i" roundup.egg-info/SOURCES.txt; done | \
      uniq -c | sort -n

    any file with a count of 2 or more needs to be removed from
    MANIFEST.in and possibly cleaned out of the build tree.
10b: if you added/removed files rebuild starting at step 6a.

11. Unpack the new tarball created in dist/roundup-<version>.tar.gz
    file in /tmp then
      a) run tests using installed pytest run under python2 and
         python3. (python2 -m pytest test/; python3 -m pytest test/)
      b) demo.py
    with all available Python versions.
11a. (TBD how to test wheel binary distribution before uploading.)

12. Assuming all is well commit and tag the release in the version-control
    system.
      a) hg commit ... # commit any edits from steps 1-5
      b) hg tag 2.1.0  # use right version. Should create/commit a changeset
      c) hg push       # update main repo
      d) hg sum        # verify that the tag shows up

13. Upload source distribution to PyPI - requires you sign up for a
    pypi account and be added as a maintainer to roundup. Ask existing
    maintainer for access. You can do this two ways:

      python3 setup.py sdist upload --repository pypi

    which rebuilds the source distribution tarball and uploads it.
    This means that you have uploaded something that is not tested,
    also it does not have a gpg signature. It should be the same as
    the tarball you tested but....

    A better way to do this is to use twine (pip install twine).
    You need to sign the tarball. This can be done with:

      cd dist
      gpg --detach-sign --armor -u 1F2DD0CB756A76D8 <filename>.tgz
      
    you should be prompted to use the roundup release key. If not you
    can add  --local=roundup-devel@lists.sourceforge.net.
    This will create a file by the name <filename>.tgz.asc.
    The original directions used twine to upload the tarball and the
    signature, but as of May 2023, PyPI no longer accepts signature
    files.

    So we publish the signature as part of the website. Move the file
    to the website/www/signatures directory. Commit the .asc signature
    file to mercurial. Add a new list item at the start of the
    signature list in doc/security.txt (look for the word multicol).

    Use twine to upload the distribution tarball. E.G.

       twine upload --repository pypi <filename>.tar.gz

    The distribution file should appear on
    https://pypi.python.org/pypi/roundup in no time. If you are using
    python older than 2.7.13 you need a .pypirc shown below since the
    URL has changed.

    You can also use twine to upload the .whl (wheel) format
    distributions (if created). Follow the directions for generating
    the gpg asc files and place the .whl.asc in the signature
    directory.

14. Refresh website.
      website/README.txt
      https://www.roundup-tracker.org/ should state that the stable
      version is the one that you released.
      https://www.roundup-tracker.org/docs.html should also match the
      released version (or at least the major non pre-release
      1.x/2.x version).

15. Send doc/announcement.txt to python-announce@python.org,
    roundup-users@lists.sourceforge.net,
    roundup-devel@lists.sourceforge.net, and lwn@lwn.net.
15b. Update entry on https://freshcode.club/projects/roundup-tracker
15c. Update entries for fossies by emailing announcement to
     announce@fossies.org
15d. Update entry on https://directory.fsf.org/wiki/Roundup.

16. Change the release version on the GitHub section at
    https://wiki.roundup-tracker.org/CiTestingEnvironment
16b. Update release info on wikipedia:
     https://en.wikipedia.org/wiki/Roundup_(issue_tracker)

     https://en.wikipedia.org/wiki/Comparison_of_issue-tracking_systems

17 Push release docker image to dockerhub
17a. install docker
17b. run: (issues, how to release a version e.g. to update alpine for
           security issues. Currently thinking that release tag is
	   rounduptracker/roundup:2.2.0-1, -2 etc? Then add a tag
	   rounduptracker/roundup:2.2.0 that moves to always tag
	   the latest -N release??)

        docker build -t rounduptracker/roundup:2.2.0 \
           --build-arg="source=pypi" -f scripts/Docker/Dockerfile .

     to create the docker image. *Change 2.2.0 to current version*
     Always use the exact release tag.
17c. vulnerability scan local image using:

        docker run --rm --volume \
	/var/run/docker.sock:/var/run/docker.sock \
            --name Grype anchore/grype:latest rounduptracker/roundup:2.2.0

     should report no vulnerabilities (note match version with current
     build)

     Also can scan (optionally) using trivy:

        docker run --rm --volume \
	/var/run/docker.sock:/var/run/docker.sock \
          --name trivy aquasec/trivy:latest image rounduptracker/roundup:2.2.0

     You may need to explicitly update/refresh the scanners with:
     "docker pull anchore/grype:latest" and similarly for
     aquasec/trivy if used.
17d. test roundup in demo mode:

        docker run -it --rm -p 8917:8080 \
         -v $PWD/tracker:/usr/src/app/tracker \
         rounduptracker/roundup:2.2.0 demo

17e. push to DockerHub login (login using 'docker login <username>'
     first and user must be member of rounduptracker org with ability
     to publish). Replace -N with the release number (e.g. -1, -2, -3...)

	 docker tag rounduptracker/roundup:2.2.0 roundup-tracker/roundup
	 docker tag rounduptracker/roundup:2.2.0 roundup-tracker/roundup:2.2.0-N
         docker push rounduptracker/roundup:2.2.0
         docker push rounduptracker/roundup:2.2.0-N
         docker push rounduptracker/roundup  # update roundup:latest

-------------

If you get errors on the upload operation, you may need the following
~/.pypirc file as well

========
[distutils]
index-servers =
    test
    pypi

[pypi]
repository: https://upload.pypi.org/legacy/
username: <your username on pypi.org here>
password: <your password here>

[test]
repository: https://test.pypi.org/legacy/
username: <your username on test.pypi.org here>
password: <your password here>
========

-------------

==========================
GPG public key operations.
==========================

LIST IN KEYRING
===============

$ gpg --list-keys -a roundup-devel@lists.sourceforge.net

   pub   rsa4096 2018-07-11 [SC] [expires: 2028-07-17]
         411E354B5D1AF26125D621221F2DD0CB756A76D8
   uid           [ultimate] Roundup Team (signing key for roundup
         releases) <roundup-devel@lists.sourceforge.net>
   sub   rsa4096 2018-07-11 [E] [expires: 2028-07-17]

EXTEND EXPIRATION DATE
======================

Needs private key and passphrse for private key

$ gpg --edit-key 411E354B5D1AF26125D621221F2DD0CB756A76D8

  > expire
  [add some number of months/years to it]
  > key 1
    [ this chooses the subkey "sub" ]
  > expire
  [add some number of months/years to the sub key ]
  > save
  [ saves both keys, will need the private key and passphrase ]

EXPORT NEW KEY
==============

$ gpg --export -a roundup-devel@lists.sourceforge.net >> \
   tools/roundup.public.pgp.key

then edit roundup.public.pgp.key keeping only the last key stat starts
with: -----BEGIN PGP PUBLIC KEY BLOCK-----

Commmit new key to mercurial.