Mercurial > p > roundup > code

annotate .github/workflows/ci-test.yml @ 6851:7840464bc064

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
fix syntax error in script enable mariadb install path disable other working paths.
author John Rouillard <rouilj@ieee.org>
date Wed, 31 Aug 2022 22:17:30 -0400
parents ce3f39195a68
children 8ffb859b01dd
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
1 # merged in python-package.yml workflow
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
2
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
3 name: roundup-ci
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
4
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
5 on:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
6 push:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
7 branches: [ "master" ]
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
8 # pull_request:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
9 # branches: [ "master" ]
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
10 # schedule:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
11 # - cron '0 11 * * 1-5'
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
12
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
13 jobs:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
14 test:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
15 name: CI build test
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
16
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
17 runs-on: ubuntu-latest
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
18 # use below if running on multiple OS's.
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
19 # runs-on: ${{ matrix.os }}
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
20
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
21 strategy:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
22 fail-fast: false
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
23 matrix:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
24 # Run in all these versions of Python
6848
eafbd37354ab don't pulll gpgme directly. also reduce runs to 3.10 and 2.7
John Rouillard <rouilj@ieee.org>
parents: 6847
diff changeset
25 python-version: [ "2.7", "3.10" ]
eafbd37354ab don't pulll gpgme directly. also reduce runs to 3.10 and 2.7
John Rouillard <rouilj@ieee.org>
parents: 6847
diff changeset
26 #python-version: [ "2.7", "3.10", "3.9", "3.8", "3.6" ]
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
27 # use for multiple os or ubuntu versions
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
28 # os: [ubuntu-latest, macos-latest, windows-latest]
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
29
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
30 #env:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
31 # OS: ${{ matrix.os }}
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
32 # PYTHON: ${{ matrix.python-version }}
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
33
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
34 steps:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
35 # Checkout the latest code from the repo
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
36 - name: Checkout source
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
37 uses: actions/checkout@v3
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
38
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
39 # Setup which version of Python to use
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
40 - name: Set Up Python ${{ matrix.python-version }}
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
41 uses: actions/setup-python@v3
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
42 with:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
43 python-version: ${{ matrix.python-version }}
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
44
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
45 # Display the Python version being used
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
46 - name: Display Python version
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
47 run: python -c "import sys; print(sys.version)"
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
48
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
49 # Install the databases
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
50 - name: Install postgres
6851
7840464bc064 fix syntax error in script enable mariadb install path
John Rouillard <rouilj@ieee.org>
parents: 6850
diff changeset
51 if: ${{ false }}
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
52 run: |
6844
c0d2017ef797 add sudo for apt-get
John Rouillard <rouilj@ieee.org>
parents: 6843
diff changeset
53 sudo apt-get install postgresql
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
54 # Disable fsync for speed, don't care about data durability when testing
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
55 sudo sed -i -e '$a\fsync = off' /etc/postgresql/*/*/postgresql.conf
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
56 sudo service postgresql restart; sleep 30
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
57 # set up postgresql database
6845
bd4d573ca733 run psql as postgres user.
John Rouillard <rouilj@ieee.org>
parents: 6844
diff changeset
58 sudo -u postgres psql -c "CREATE ROLE rounduptest WITH CREATEDB LOGIN PASSWORD 'rounduptest';" -U postgres
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
59
6850
ce3f39195a68 updates and remove tab from yml.
John Rouillard <rouilj@ieee.org>
parents: 6849
diff changeset
60 - name: Install mariadb/mysql
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
61 run: |
6850
ce3f39195a68 updates and remove tab from yml.
John Rouillard <rouilj@ieee.org>
parents: 6849
diff changeset
62 sudo apt-get install mariadb
6841
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
63 # set up mysql database
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
64 sudo sed -i -e '/^\[mysqld\]/,/^\[mysql/s/^max_allowed_packet.*/max_allowed_packet = 500M/' /etc/mysql/my.cnf
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
65 cat /etc/mysql/my.cnf
6850
ce3f39195a68 updates and remove tab from yml.
John Rouillard <rouilj@ieee.org>
parents: 6849
diff changeset
66 sudo service mariadb restart
6841
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
67 mysql -u root -e 'GRANT ALL ON rounduptest.* TO rounduptest@localhost IDENTIFIED BY "rounduptest";'
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
68 # HACK: workaround mysql bug: http://bugs.mysql.com/bug.php?id=74901
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
69 # needed for test_mysql.mysqlDBTest.testFilteringSpecialChars
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
70 # plus others. Otherwise we get:
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
71 # COLLATION 'utf8_bin' is not valid for CHARACTER SET 'utf8mb4'
6850
ce3f39195a68 updates and remove tab from yml.
John Rouillard <rouilj@ieee.org>
parents: 6849
diff changeset
72 # sed -i 's/CREATE DATABASE \%s/CREATE DATABASE \%s COLLATE utf8_general_ci/' roundup/backends/back_mysql.py
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
73
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
74 - name: Update pip
6851
7840464bc064 fix syntax error in script enable mariadb install path
John Rouillard <rouilj@ieee.org>
parents: 6850
diff changeset
75 if: ${{ false }}
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
76 run: python -m pip install --upgrade pip
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
77
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
78 - name: Install db libraries
6851
7840464bc064 fix syntax error in script enable mariadb install path
John Rouillard <rouilj@ieee.org>
parents: 6850
diff changeset
79 if: ${{ false }}
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
80 run: pip install psycopg2 mysqlclient
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
81
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
82 - name: Install auxilary packages
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
83 run: |
6850
ce3f39195a68 updates and remove tab from yml.
John Rouillard <rouilj@ieee.org>
parents: 6849
diff changeset
84 sudo apt-get install swig gpgsm libgpgme-dev
6849
86c64ed14a6d defer gpg install for now.
John Rouillard <rouilj@ieee.org>
parents: 6848
diff changeset
85 pip install beautifulsoup4 brotli jinja2 markdown2 \
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
86 pyjwt pytz whoosh
6850
ce3f39195a68 updates and remove tab from yml.
John Rouillard <rouilj@ieee.org>
parents: 6849
diff changeset
87 # pip install gpg
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
88
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
89 - name: Install aux packages that need versions differences
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
90 # if zstd fails install, keep going with test, don't abort
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
91 run: |
6842
d8d4600c49eb More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6841
diff changeset
92 set -xv
6840
8ee17804086e untabify yml file.
John Rouillard <rouilj@ieee.org>
parents: 6839
diff changeset
93 pip install zstd || true
6851
7840464bc064 fix syntax error in script enable mariadb install path
John Rouillard <rouilj@ieee.org>
parents: 6850
diff changeset
94 if [[ $(python -c 'import sys; print("%d.%d.%d"%(sys.version_info.major, sys.version_info.minor, sys.version_info.micro))') != "2."* ]]; then
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
95 pip install Markdown; fi
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
96
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
97 - name: Install pytest and other packages needed for running tests
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
98 run: pip install codecov flake8 pytest pytest-cov requests
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
99
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
100 - name: Test build roundup and install locale so lang tests work.
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
101 run: |
6842
d8d4600c49eb More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6841
diff changeset
102 python setup.py install
6840
8ee17804086e untabify yml file.
John Rouillard <rouilj@ieee.org>
parents: 6839
diff changeset
103 (cd locale; make local_install; ls -lR locale/de/LC_MESSAGES)
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
104
6843
c61ab02ad067 fix syntax error name can not include :
John Rouillard <rouilj@ieee.org>
parents: 6842
diff changeset
105 - name: run flake8 - abort for syntax error, otherwise warn only
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
106 run: |
6840
8ee17804086e untabify yml file.
John Rouillard <rouilj@ieee.org>
parents: 6839
diff changeset
107 # stop the build for Python syntax errors or undefined names
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
108 flake8 roundup --count --select=E9,F63,F7,F82 --show-source --statistics
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
109 # exit-zero treats all errors as warnings.
6840
8ee17804086e untabify yml file.
John Rouillard <rouilj@ieee.org>
parents: 6839
diff changeset
110 # The GitHub editor is 127 chars wide
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
111 flake8 roundup --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
112
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
113 # Run the tests using pytest with test files in tests directory.
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
114 - name: Run tests
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
115 run: pytest -v tests
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
116
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
117 - name: Upload coverage to Codecov
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
118 # see: https://github.com/codecov/codecov-action#usage
6840
8ee17804086e untabify yml file.
John Rouillard <rouilj@ieee.org>
parents: 6839
diff changeset
119 uses: codecov/codecov-action@v3
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
120 with:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
121 verbose: true
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
122
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
123 - name: test build_doc
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
124 run: |
6842
d8d4600c49eb More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6841
diff changeset
125 python ./setup.py build_doc
Roundup Issue Tracker: http://roundup-tracker.org/