Mercurial > p > roundup > code

annotate .github/workflows/ci-test.yml @ 6858:9309493b70c4

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
more
author John Rouillard <rouilj@ieee.org>
date Wed, 31 Aug 2022 23:05:18 -0400
parents a346b5d23480
children ebdadf3b2653
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
1 # merged in python-package.yml workflow
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
2
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
3 name: roundup-ci
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
4
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
5 on:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
6 push:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
7 branches: [ "master" ]
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
8 # pull_request:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
9 # branches: [ "master" ]
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
10 # schedule:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
11 # - cron '0 11 * * 1-5'
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
12
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
13 jobs:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
14 test:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
15 name: CI build test
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
16
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
17 runs-on: ubuntu-latest
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
18 # use below if running on multiple OS's.
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
19 # runs-on: ${{ matrix.os }}
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
20
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
21 strategy:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
22 fail-fast: false
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
23 matrix:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
24 # Run in all these versions of Python
6848
eafbd37354ab don't pulll gpgme directly. also reduce runs to 3.10 and 2.7
John Rouillard <rouilj@ieee.org>
parents: 6847
diff changeset
25 python-version: [ "2.7", "3.10" ]
eafbd37354ab don't pulll gpgme directly. also reduce runs to 3.10 and 2.7
John Rouillard <rouilj@ieee.org>
parents: 6847
diff changeset
26 #python-version: [ "2.7", "3.10", "3.9", "3.8", "3.6" ]
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
27 # use for multiple os or ubuntu versions
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
28 # os: [ubuntu-latest, macos-latest, windows-latest]
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
29
6858
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
30 env:
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
31 # OS: ${{ matrix.os }}
6858
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
32 PYTHON_VERSION: ${{ matrix.python-version }}
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
33
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
34 steps:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
35 # Checkout the latest code from the repo
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
36 - name: Checkout source
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
37 uses: actions/checkout@v3
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
38
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
39 # Setup which version of Python to use
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
40 - name: Set Up Python ${{ matrix.python-version }}
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
41 uses: actions/setup-python@v3
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
42 with:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
43 python-version: ${{ matrix.python-version }}
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
44
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
45 # Display the Python version being used
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
46 - name: Display Python version
6858
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
47 run: python -c "import sys; print(sys.version)"; echo $PYTHON_VERSION
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
48
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
49 # Install the databases
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
50 - name: Install postgres
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
51 run: |
6844
c0d2017ef797 add sudo for apt-get
John Rouillard <rouilj@ieee.org>
parents: 6843
diff changeset
52 sudo apt-get install postgresql
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
53 # Disable fsync for speed, don't care about data durability when testing
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
54 sudo sed -i -e '$a\fsync = off' /etc/postgresql/*/*/postgresql.conf
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
55 sudo service postgresql restart; sleep 30
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
56 # set up postgresql database
6845
bd4d573ca733 run psql as postgres user.
John Rouillard <rouilj@ieee.org>
parents: 6844
diff changeset
57 sudo -u postgres psql -c "CREATE ROLE rounduptest WITH CREATEDB LOGIN PASSWORD 'rounduptest';" -U postgres
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
58
6852
8ffb859b01dd more fixes.
John Rouillard <rouilj@ieee.org>
parents: 6851
diff changeset
59 - name: Install mysql/mariadb
6853
1f12ca2c409c mysql off again
John Rouillard <rouilj@ieee.org>
parents: 6852
diff changeset
60 if: ${{ false }}
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
61 run: |
6852
8ffb859b01dd more fixes.
John Rouillard <rouilj@ieee.org>
parents: 6851
diff changeset
62 sudo apt-get install mysql-server mysql-client
6841
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
63 # set up mysql database
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
64 sudo sed -i -e '/^\[mysqld\]/,/^\[mysql/s/^max_allowed_packet.*/max_allowed_packet = 500M/' /etc/mysql/my.cnf
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
65 cat /etc/mysql/my.cnf
6852
8ffb859b01dd more fixes.
John Rouillard <rouilj@ieee.org>
parents: 6851
diff changeset
66 sudo service mysql restart
6841
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
67 mysql -u root -e 'GRANT ALL ON rounduptest.* TO rounduptest@localhost IDENTIFIED BY "rounduptest";'
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
68 # HACK: workaround mysql bug: http://bugs.mysql.com/bug.php?id=74901
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
69 # needed for test_mysql.mysqlDBTest.testFilteringSpecialChars
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
70 # plus others. Otherwise we get:
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
71 # COLLATION 'utf8_bin' is not valid for CHARACTER SET 'utf8mb4'
6850
ce3f39195a68 updates and remove tab from yml.
John Rouillard <rouilj@ieee.org>
parents: 6849
diff changeset
72 # sed -i 's/CREATE DATABASE \%s/CREATE DATABASE \%s COLLATE utf8_general_ci/' roundup/backends/back_mysql.py
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
73
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
74 - name: Update pip
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
75 run: python -m pip install --upgrade pip
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
76
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
77 - name: Install db libraries
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
78 run: pip install psycopg2 mysqlclient
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
79
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
80 - name: Install auxilary packages
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
81 run: |
6850
ce3f39195a68 updates and remove tab from yml.
John Rouillard <rouilj@ieee.org>
parents: 6849
diff changeset
82 sudo apt-get install swig gpgsm libgpgme-dev
6849
86c64ed14a6d defer gpg install for now.
John Rouillard <rouilj@ieee.org>
parents: 6848
diff changeset
83 pip install beautifulsoup4 brotli jinja2 markdown2 \
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
84 pyjwt pytz whoosh
6850
ce3f39195a68 updates and remove tab from yml.
John Rouillard <rouilj@ieee.org>
parents: 6849
diff changeset
85 # pip install gpg
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
86
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
87 - name: Install aux packages that need versions differences
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
88 # if zstd fails install, keep going with test, don't abort
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
89 run: |
6842
d8d4600c49eb More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6841
diff changeset
90 set -xv
6840
8ee17804086e untabify yml file.
John Rouillard <rouilj@ieee.org>
parents: 6839
diff changeset
91 pip install zstd || true
6851
7840464bc064 fix syntax error in script enable mariadb install path
John Rouillard <rouilj@ieee.org>
parents: 6850
diff changeset
92 if [[ $(python -c 'import sys; print("%d.%d.%d"%(sys.version_info.major, sys.version_info.minor, sys.version_info.micro))') != "2."* ]]; then
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
93 pip install Markdown; fi
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
94
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
95 - name: Install pytest and other packages needed for running tests
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
96 run: pip install codecov flake8 pytest pytest-cov requests
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
97
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
98 - name: Test build roundup and install locale so lang tests work.
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
99 run: |
6855
30815d9bd95c fix tab
John Rouillard <rouilj@ieee.org>
parents: 6854
diff changeset
100 sudo apt-get install gettext
6854
4f115313e262 install gettext for locale
John Rouillard <rouilj@ieee.org>
parents: 6853
diff changeset
101 python setup.py build
6840
8ee17804086e untabify yml file.
John Rouillard <rouilj@ieee.org>
parents: 6839
diff changeset
102 (cd locale; make local_install; ls -lR locale/de/LC_MESSAGES)
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
103
6843
c61ab02ad067 fix syntax error name can not include :
John Rouillard <rouilj@ieee.org>
parents: 6842
diff changeset
104 - name: run flake8 - abort for syntax error, otherwise warn only
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
105 run: |
6840
8ee17804086e untabify yml file.
John Rouillard <rouilj@ieee.org>
parents: 6839
diff changeset
106 # stop the build for Python syntax errors or undefined names
6857
a346b5d23480 flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6855
diff changeset
107 # talgettext is a utility function ignore it.
a346b5d23480 flake8 fixes.
John Rouillard <rouilj@ieee.org>
parents: 6855
diff changeset
108 flake8 roundup --count --select=E9,F63,F7,F82 --show-source --statistics --extend-exclude talgettext.py
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
109 # exit-zero treats all errors as warnings.
6840
8ee17804086e untabify yml file.
John Rouillard <rouilj@ieee.org>
parents: 6839
diff changeset
110 # The GitHub editor is 127 chars wide
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
111 flake8 roundup --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
112
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
113 # Run the tests using pytest with test files in tests directory.
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
114 - name: Run tests
6858
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
115 run: |
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
116 if [[ $(python -c 'import sys; print("%d.%d.%d"%(sys.version_info.major, sys.version_info.minor, sys.version_info.micro))') != "2."* ]]; then
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
117 pytest -r a \
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
118 -W default \
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
119 -W "ignore:SelectableGroups:DeprecationWarning" \
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
120 -W "ignore:the imp module:DeprecationWarning:gpg.gpgme:15" \
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
121 -W "ignore:'U' mode::docutils.io" \
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
122 -W "ignore:unclosed:ResourceWarning:roundup.roundup.demo" \
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
123 -W "ignore:unclosed file:ResourceWarning:enum" \
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
124 -v --maxfail=5 test/ --cov=roundup
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
125 else
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
126 # python2 case
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
127 pytest -v -r a --maxfail=5 test/ --cov=roundup
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
128 fi
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
129
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
130 - name: Upload coverage to Codecov
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
131 # see: https://github.com/codecov/codecov-action#usage
6840
8ee17804086e untabify yml file.
John Rouillard <rouilj@ieee.org>
parents: 6839
diff changeset
132 uses: codecov/codecov-action@v3
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
133 with:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
134 verbose: true
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
135
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
136 - name: test build_doc
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
137 run: |
6842
d8d4600c49eb More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6841
diff changeset
138 python ./setup.py build_doc
Roundup Issue Tracker: http://roundup-tracker.org/