Mercurial > p > roundup > code

annotate .github/workflows/ci-test.yml @ 8323:32a57cde072f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
chore(ci): issue2551368 use new gpg for all releases. I was the new version only for 3.10 and newer running on ubuntu 24.04. Paul said it should work on 3.7 and 3.8 which runn on unbuntu 22.04. So enable it and see if it works.
author John Rouillard <rouilj@ieee.org>
date Wed, 04 Jun 2025 19:20:27 -0400
parents 71e961941be6
children 375c9f63f877
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
1 # merged in python-package.yml workflow
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
2
7055
83dc71b3211f add web references used to build github ci file.
John Rouillard <rouilj@ieee.org>
parents: 6955
diff changeset
3 # reference docs:
83dc71b3211f add web references used to build github ci file.
John Rouillard <rouilj@ieee.org>
parents: 6955
diff changeset
4 # https://blog.deepjyoti30.dev/tests-github-python
83dc71b3211f add web references used to build github ci file.
John Rouillard <rouilj@ieee.org>
parents: 6955
diff changeset
5 # https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
7097
d9b153732bb1 add new reference.
John Rouillard <rouilj@ieee.org>
parents: 7055
diff changeset
6 # https://github.com/pypa/twine/blob/main/.github/workflows/main.yml
d9b153732bb1 add new reference.
John Rouillard <rouilj@ieee.org>
parents: 7055
diff changeset
7
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
8 name: roundup-ci
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
9
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
10 on:
6887
dc7b886c42e4 run on any branch push except maint-1.6
John Rouillard <rouilj@ieee.org>
parents: 6886
diff changeset
11 push:
6904
92b64fd3341f cleanup, comments ready for production.
John Rouillard <rouilj@ieee.org>
parents: 6903
diff changeset
12 # skip if github.ref is 'refs/heads/maint-1.6'
92b64fd3341f cleanup, comments ready for production.
John Rouillard <rouilj@ieee.org>
parents: 6903
diff changeset
13 # aka github.ref_name of 'maint-1.6'
92b64fd3341f cleanup, comments ready for production.
John Rouillard <rouilj@ieee.org>
parents: 6903
diff changeset
14 # see https://github.com/orgs/community/discussions/26253
92b64fd3341f cleanup, comments ready for production.
John Rouillard <rouilj@ieee.org>
parents: 6903
diff changeset
15 # for mechanism to control matrix based on branch
6887
dc7b886c42e4 run on any branch push except maint-1.6
John Rouillard <rouilj@ieee.org>
parents: 6886
diff changeset
16 branches: [ "*", '!maint-1.6' ]
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
17 # pull_request:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
18 # branches: [ "master" ]
6880
222e84d12bed schedule build 7PM on first of month; commented out docker tst
John Rouillard <rouilj@ieee.org>
parents: 6879
diff changeset
19 schedule:
222e84d12bed schedule build 7PM on first of month; commented out docker tst
John Rouillard <rouilj@ieee.org>
parents: 6879
diff changeset
20 # monthly build/check
6885
cc9bd0dd39fc change time again. also build on all pushes
John Rouillard <rouilj@ieee.org>
parents: 6884
diff changeset
21 - cron: '23 17 1 * *'
7849
3fe2e7b3df58 test(github-ci): add method to ssh into a CI test environment
John Rouillard <rouilj@ieee.org>
parents: 7828
diff changeset
22 workflow_dispatch:
3fe2e7b3df58 test(github-ci): add method to ssh into a CI test environment
John Rouillard <rouilj@ieee.org>
parents: 7828
diff changeset
23 inputs:
3fe2e7b3df58 test(github-ci): add method to ssh into a CI test environment
John Rouillard <rouilj@ieee.org>
parents: 7828
diff changeset
24 debug_enabled:
3fe2e7b3df58 test(github-ci): add method to ssh into a CI test environment
John Rouillard <rouilj@ieee.org>
parents: 7828
diff changeset
25 type: boolean
3fe2e7b3df58 test(github-ci): add method to ssh into a CI test environment
John Rouillard <rouilj@ieee.org>
parents: 7828
diff changeset
26 description: 'Run the build with tmate debugging enabled (https://github.com/marketplace/actions/debugging-with-tmate)'
3fe2e7b3df58 test(github-ci): add method to ssh into a CI test environment
John Rouillard <rouilj@ieee.org>
parents: 7828
diff changeset
27 required: false
3fe2e7b3df58 test(github-ci): add method to ssh into a CI test environment
John Rouillard <rouilj@ieee.org>
parents: 7828
diff changeset
28 default: false
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
29
6914
6010c20dc104 add mock package for testing
John Rouillard <rouilj@ieee.org>
parents: 6904
diff changeset
30 # GITHUB_TOKEN only has read repo context.
6010c20dc104 add mock package for testing
John Rouillard <rouilj@ieee.org>
parents: 6904
diff changeset
31 permissions:
6010c20dc104 add mock package for testing
John Rouillard <rouilj@ieee.org>
parents: 6904
diff changeset
32 contents: read
6010c20dc104 add mock package for testing
John Rouillard <rouilj@ieee.org>
parents: 6904
diff changeset
33
6953
301f352b5d63 Only do one build at a time. Cancel older in progress builds.
John Rouillard <rouilj@ieee.org>
parents: 6952
diff changeset
34 concurrency:
301f352b5d63 Only do one build at a time. Cancel older in progress builds.
John Rouillard <rouilj@ieee.org>
parents: 6952
diff changeset
35 group: ${{ github.workflow }}-${{ github.ref }}
6954
22177f3ea206 fix indentation.
John Rouillard <rouilj@ieee.org>
parents: 6953
diff changeset
36 cancel-in-progress: true
6953
301f352b5d63 Only do one build at a time. Cancel older in progress builds.
John Rouillard <rouilj@ieee.org>
parents: 6952
diff changeset
37
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
38 jobs:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
39 test:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
40 name: CI build test
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
41
6891
be310c5f866e another trial with exclude and include
John Rouillard <rouilj@ieee.org>
parents: 6890
diff changeset
42 #runs-on: ubuntu-latest
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
43 # use below if running on multiple OS's.
6891
be310c5f866e another trial with exclude and include
John Rouillard <rouilj@ieee.org>
parents: 6890
diff changeset
44 runs-on: ${{ matrix.os }}
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
45
7194
8dc5b3739367 Prevent github actions from running if commit includes 'no-github-ci'
John Rouillard <rouilj@ieee.org>
parents: 7187
diff changeset
46 if: "!contains(github.event.head_commit.message, 'no-github-ci')"
8dc5b3739367 Prevent github actions from running if commit includes 'no-github-ci'
John Rouillard <rouilj@ieee.org>
parents: 7187
diff changeset
47
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
48 strategy:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
49 fail-fast: false
6878
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
50 max-parallel: 4
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
51 matrix:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
52 # Run in all these versions of Python
7107
8e2219abbde3 infra: support new ubuntu 22.04 runners.
John Rouillard <rouilj@ieee.org>
parents: 7105
diff changeset
53 python-version:
7510
8ae4b6498345 Disable python 2.7 tests on github.
John Rouillard <rouilj@ieee.org>
parents: 7509
diff changeset
54 # - "2.7"
8118
c88ad93f0e77 issue2551339 - psycopg2 on 3.13 github runner crashes on import
John Rouillard <rouilj@ieee.org>
parents: 8117
diff changeset
55 - "3.13"
c88ad93f0e77 issue2551339 - psycopg2 on 3.13 github runner crashes on import
John Rouillard <rouilj@ieee.org>
parents: 8117
diff changeset
56 # - 3.6 run via include on ubuntu 20.04
7894
7b0a143ac806 test: disable Python 3.13 CI xapian build. Fails with compile error.
John Rouillard <rouilj@ieee.org>
parents: 7893
diff changeset
57 # - "3.7"
8140
0bd23d5a8122 test changes for 24.04 ubuntu github action migration
John Rouillard <rouilj@ieee.org>
parents: 8118
diff changeset
58 # - "3.8" run via include for ubuntu-22.04
7894
7b0a143ac806 test: disable Python 3.13 CI xapian build. Fails with compile error.
John Rouillard <rouilj@ieee.org>
parents: 7893
diff changeset
59 # - "3.9"
7b0a143ac806 test: disable Python 3.13 CI xapian build. Fails with compile error.
John Rouillard <rouilj@ieee.org>
parents: 7893
diff changeset
60 - "3.10"
8152
0d2a95352679 drop testing python 3.11 from matrix.
John Rouillard <rouilj@ieee.org>
parents: 8151
diff changeset
61 # - "3.11"
8118
c88ad93f0e77 issue2551339 - psycopg2 on 3.13 github runner crashes on import
John Rouillard <rouilj@ieee.org>
parents: 8117
diff changeset
62 - "3.12"
6904
92b64fd3341f cleanup, comments ready for production.
John Rouillard <rouilj@ieee.org>
parents: 6903
diff changeset
63
6878
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
64 # use for multiple os or ubuntu versions
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
65 #os: [ubuntu-latest, macos-latest, windows-latest]
7107
8e2219abbde3 infra: support new ubuntu 22.04 runners.
John Rouillard <rouilj@ieee.org>
parents: 7105
diff changeset
66 # ubuntu latest 22.04 12/2022
8140
0bd23d5a8122 test changes for 24.04 ubuntu github action migration
John Rouillard <rouilj@ieee.org>
parents: 8118
diff changeset
67 # ubuntu latest 24.04 12/2024
8222
54dfda1c4fe5 issue251338 - xapian doesn't build in CI for 3.13 python
John Rouillard <rouilj@ieee.org>
parents: 8158
diff changeset
68 os: [ubuntu-latest]
6904
92b64fd3341f cleanup, comments ready for production.
John Rouillard <rouilj@ieee.org>
parents: 6903
diff changeset
69
6878
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
70 # if the ones above fail. fail the build
7540
aa79c91f913a try unquoted string.
John Rouillard <rouilj@ieee.org>
parents: 7539
diff changeset
71 experimental: [ false ]
6904
92b64fd3341f cleanup, comments ready for production.
John Rouillard <rouilj@ieee.org>
parents: 6903
diff changeset
72
6888
f4705aba3503 add ubuntu-22.04 for 3.10 and 3.8
John Rouillard <rouilj@ieee.org>
parents: 6887
diff changeset
73 include:
7893
ff5cadffa8f0 chore: change spec for 3.13.
John Rouillard <rouilj@ieee.org>
parents: 7892
diff changeset
74 # example: if 3.13 fails the jobs still succeeds
ff5cadffa8f0 chore: change spec for 3.13.
John Rouillard <rouilj@ieee.org>
parents: 7892
diff changeset
75 # allow-prereleases in setup-python allows alpha/beta
ff5cadffa8f0 chore: change spec for 3.13.
John Rouillard <rouilj@ieee.org>
parents: 7892
diff changeset
76 # releases to run
8118
c88ad93f0e77 issue2551339 - psycopg2 on 3.13 github runner crashes on import
John Rouillard <rouilj@ieee.org>
parents: 8117
diff changeset
77 #- python-version: 3.13
c88ad93f0e77 issue2551339 - psycopg2 on 3.13 github runner crashes on import
John Rouillard <rouilj@ieee.org>
parents: 8117
diff changeset
78 # os: ubuntu-22.04
c88ad93f0e77 issue2551339 - psycopg2 on 3.13 github runner crashes on import
John Rouillard <rouilj@ieee.org>
parents: 8117
diff changeset
79 # experimental: true
7534
9b9eb81abef7 skip travisci build
John Rouillard <rouilj@ieee.org>
parents: 7533
diff changeset
80
8309
81ca7ceb71db build: issue2551397: remove support for python 3.6
John Rouillard <rouilj@ieee.org>
parents: 8223
diff changeset
81 # 3.7 not available on new 22.04 runners, so run on 22.04 ubuntu
81ca7ceb71db build: issue2551397: remove support for python 3.6
John Rouillard <rouilj@ieee.org>
parents: 8223
diff changeset
82 - python-version: 3.7
81ca7ceb71db build: issue2551397: remove support for python 3.6
John Rouillard <rouilj@ieee.org>
parents: 8223
diff changeset
83 os: ubuntu-22.04
6904
92b64fd3341f cleanup, comments ready for production.
John Rouillard <rouilj@ieee.org>
parents: 6903
diff changeset
84
8140
0bd23d5a8122 test changes for 24.04 ubuntu github action migration
John Rouillard <rouilj@ieee.org>
parents: 8118
diff changeset
85 # 3.8 not available on new 24.04 runners, so run on 22.04 ubuntu
0bd23d5a8122 test changes for 24.04 ubuntu github action migration
John Rouillard <rouilj@ieee.org>
parents: 8118
diff changeset
86 - python-version: 3.8
0bd23d5a8122 test changes for 24.04 ubuntu github action migration
John Rouillard <rouilj@ieee.org>
parents: 8118
diff changeset
87 os: ubuntu-22.04
0bd23d5a8122 test changes for 24.04 ubuntu github action migration
John Rouillard <rouilj@ieee.org>
parents: 8118
diff changeset
88
8223
cfd8eb705c2f test: fix syntax
John Rouillard <rouilj@ieee.org>
parents: 8222
diff changeset
89 #exclude:
8222
54dfda1c4fe5 issue251338 - xapian doesn't build in CI for 3.13 python
John Rouillard <rouilj@ieee.org>
parents: 8158
diff changeset
90 # skip all python versions on explicit 20.04/24.04 unless
8140
0bd23d5a8122 test changes for 24.04 ubuntu github action migration
John Rouillard <rouilj@ieee.org>
parents: 8118
diff changeset
91 # explicitly included
8222
54dfda1c4fe5 issue251338 - xapian doesn't build in CI for 3.13 python
John Rouillard <rouilj@ieee.org>
parents: 8158
diff changeset
92 #- os: ubuntu-20.04
54dfda1c4fe5 issue251338 - xapian doesn't build in CI for 3.13 python
John Rouillard <rouilj@ieee.org>
parents: 8158
diff changeset
93 #- os: ubuntu-24.04
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
94
8150
911c6b0fbaf7 cleanup - 24.04 ubuntu github action migration
John Rouillard <rouilj@ieee.org>
parents: 8149
diff changeset
95 # disable when testing ubuntu-24.04 to speed up
911c6b0fbaf7 cleanup - 24.04 ubuntu github action migration
John Rouillard <rouilj@ieee.org>
parents: 8149
diff changeset
96 # runs.
911c6b0fbaf7 cleanup - 24.04 ubuntu github action migration
John Rouillard <rouilj@ieee.org>
parents: 8149
diff changeset
97 # - os: ubuntu-latest
911c6b0fbaf7 cleanup - 24.04 ubuntu github action migration
John Rouillard <rouilj@ieee.org>
parents: 8149
diff changeset
98
7541
72e1ee3a46c0 one last try move continue-on-error after all matrix defs
John Rouillard <rouilj@ieee.org>
parents: 7540
diff changeset
99 # run the finalizer for coveralls even if one or more
72e1ee3a46c0 one last try move continue-on-error after all matrix defs
John Rouillard <rouilj@ieee.org>
parents: 7540
diff changeset
100 # experimental matrix runs fail.
7542
3b67f0e7fe48 I give up.
John Rouillard <rouilj@ieee.org>
parents: 7541
diff changeset
101 # moving it above strategy produces unexpected value false
3b67f0e7fe48 I give up.
John Rouillard <rouilj@ieee.org>
parents: 7541
diff changeset
102 # moving it below (here) produces unexpected value ''.
3b67f0e7fe48 I give up.
John Rouillard <rouilj@ieee.org>
parents: 7541
diff changeset
103 # continue-on-error: ${{ matrix.experimental }}
7541
72e1ee3a46c0 one last try move continue-on-error after all matrix defs
John Rouillard <rouilj@ieee.org>
parents: 7540
diff changeset
104
6858
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
105 env:
6864
7e907d97deb6 redis, gpg, more myql debugging
John Rouillard <rouilj@ieee.org>
parents: 6863
diff changeset
106 # get colorized pytest output even without a controlling tty
7e907d97deb6 redis, gpg, more myql debugging
John Rouillard <rouilj@ieee.org>
parents: 6863
diff changeset
107 PYTEST_ADDOPTS: "--color=yes"
6859
ebdadf3b2653 fixes and re-enable mysql.
John Rouillard <rouilj@ieee.org>
parents: 6858
diff changeset
108 # OS: ${{ matrix.os }}
6860
69928bcce59d fix env again.
John Rouillard <rouilj@ieee.org>
parents: 6859
diff changeset
109 PYTHON_VERSION: ${{ matrix.python-version }}
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
110
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
111 steps:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
112 # Checkout the latest code from the repo
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
113 - name: Checkout source
6878
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
114 # example directives:
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
115 # disable step
6873
8dab742710b6 null pw??
John Rouillard <rouilj@ieee.org>
parents: 6872
diff changeset
116 # if: {{ false }}
6878
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
117 # continue running if step fails
6874
70a78f281179 fix bad tab.
John Rouillard <rouilj@ieee.org>
parents: 6873
diff changeset
118 # continue-on-error: true
7729
edd93fa4a5ec chore: update actions in gihub workflows.
John Rouillard <rouilj@ieee.org>
parents: 7721
diff changeset
119 uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
120
6878
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
121 # Setup version of Python to use
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
122 - name: Set Up Python ${{ matrix.python-version }}
7892
3f55b5e33a60 chore: add 3.13-alpha to CI and make 3.12 failures fail the build
John Rouillard <rouilj@ieee.org>
parents: 7891
diff changeset
123 uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5.1.0
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
124 with:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
125 python-version: ${{ matrix.python-version }}
7272
2e48fb118772 Allow fallback to pre-release versions if GA version not released.
John Rouillard <rouilj@ieee.org>
parents: 7271
diff changeset
126 allow-prereleases: true
6879
5ba79dd2f31c add codecov token
John Rouillard <rouilj@ieee.org>
parents: 6878
diff changeset
127 cache: 'pip'
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
128
7415
f636c3c5333e Move setuptools install location and print version
John Rouillard <rouilj@ieee.org>
parents: 7414
diff changeset
129 - name: Install build tools - setuptools
f636c3c5333e Move setuptools install location and print version
John Rouillard <rouilj@ieee.org>
parents: 7414
diff changeset
130 run: pip install setuptools
f636c3c5333e Move setuptools install location and print version
John Rouillard <rouilj@ieee.org>
parents: 7414
diff changeset
131
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
132 # Display the Python version being used
6920
b3fb615b81c4 get info about sqlite version
John Rouillard <rouilj@ieee.org>
parents: 6919
diff changeset
133 - name: Display Python and key module versions
b3fb615b81c4 get info about sqlite version
John Rouillard <rouilj@ieee.org>
parents: 6919
diff changeset
134 run: |
b3fb615b81c4 get info about sqlite version
John Rouillard <rouilj@ieee.org>
parents: 6919
diff changeset
135 python -c "import sys; print('python version: ', sys.version)"
7274
5ecaebc08447 remove obsolete sqlite3.version
John Rouillard <rouilj@ieee.org>
parents: 7272
diff changeset
136 python -c "import sqlite3; print('sqlite version: ', sqlite3.sqlite_version)"
7415
f636c3c5333e Move setuptools install location and print version
John Rouillard <rouilj@ieee.org>
parents: 7414
diff changeset
137 python -c "import setuptools; print('setuptools version: ', setuptools.__version__);"
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
138
7763
a8891a0e10cb chore: optimize build file make it error sooner on flake8 tests
John Rouillard <rouilj@ieee.org>
parents: 7742
diff changeset
139 - name: Update pip
a8891a0e10cb chore: optimize build file make it error sooner on flake8 tests
John Rouillard <rouilj@ieee.org>
parents: 7742
diff changeset
140 run: python -m pip install --upgrade pip
a8891a0e10cb chore: optimize build file make it error sooner on flake8 tests
John Rouillard <rouilj@ieee.org>
parents: 7742
diff changeset
141
7823
5ac9253d2a5f test: add comment that ci tests don't need pytest-env.
John Rouillard <rouilj@ieee.org>
parents: 7763
diff changeset
142 # note pytest-env is not needed for redis password as there is
5ac9253d2a5f test: add comment that ci tests don't need pytest-env.
John Rouillard <rouilj@ieee.org>
parents: 7763
diff changeset
143 # no password on CI's redis.
7763
a8891a0e10cb chore: optimize build file make it error sooner on flake8 tests
John Rouillard <rouilj@ieee.org>
parents: 7742
diff changeset
144 - name: Install pytest and other packages needed for running tests
7828
3c514819c1b7 test: add hypothesis
John Rouillard <rouilj@ieee.org>
parents: 7823
diff changeset
145 run: pip install flake8 hypothesis mock pytest pytest-cov requests sphinx-tabs
7763
a8891a0e10cb chore: optimize build file make it error sooner on flake8 tests
John Rouillard <rouilj@ieee.org>
parents: 7742
diff changeset
146
7852
9e686e3f39c1 test(githubci): incrase duration of tmate shell on test runner
John Rouillard <rouilj@ieee.org>
parents: 7850
diff changeset
147 # https://github.com/mxschmitt/action-tmate
9e686e3f39c1 test(githubci): incrase duration of tmate shell on test runner
John Rouillard <rouilj@ieee.org>
parents: 7850
diff changeset
148 # allow remote ssh into the CI container. I need this to debug
9e686e3f39c1 test(githubci): incrase duration of tmate shell on test runner
John Rouillard <rouilj@ieee.org>
parents: 7850
diff changeset
149 # some xfail cases
9e686e3f39c1 test(githubci): incrase duration of tmate shell on test runner
John Rouillard <rouilj@ieee.org>
parents: 7850
diff changeset
150 - name: Setup tmate session
9e686e3f39c1 test(githubci): incrase duration of tmate shell on test runner
John Rouillard <rouilj@ieee.org>
parents: 7850
diff changeset
151 uses: mxschmitt/action-tmate@v3
9e686e3f39c1 test(githubci): incrase duration of tmate shell on test runner
John Rouillard <rouilj@ieee.org>
parents: 7850
diff changeset
152 if: ${{ github.event_name == 'workflow_dispatch' && inputs.debug_enabled }}
9e686e3f39c1 test(githubci): incrase duration of tmate shell on test runner
John Rouillard <rouilj@ieee.org>
parents: 7850
diff changeset
153 timeout-minutes: 10
9e686e3f39c1 test(githubci): incrase duration of tmate shell on test runner
John Rouillard <rouilj@ieee.org>
parents: 7850
diff changeset
154 with:
9e686e3f39c1 test(githubci): incrase duration of tmate shell on test runner
John Rouillard <rouilj@ieee.org>
parents: 7850
diff changeset
155 limit-access-to-actor: true
9e686e3f39c1 test(githubci): incrase duration of tmate shell on test runner
John Rouillard <rouilj@ieee.org>
parents: 7850
diff changeset
156
7763
a8891a0e10cb chore: optimize build file make it error sooner on flake8 tests
John Rouillard <rouilj@ieee.org>
parents: 7742
diff changeset
157 - name: run flake8 - abort for syntax error, otherwise warn only
a8891a0e10cb chore: optimize build file make it error sooner on flake8 tests
John Rouillard <rouilj@ieee.org>
parents: 7742
diff changeset
158 run: |
a8891a0e10cb chore: optimize build file make it error sooner on flake8 tests
John Rouillard <rouilj@ieee.org>
parents: 7742
diff changeset
159 # stop the build for Python syntax errors or undefined names
a8891a0e10cb chore: optimize build file make it error sooner on flake8 tests
John Rouillard <rouilj@ieee.org>
parents: 7742
diff changeset
160 # talgettext is a utility function ignore it.
a8891a0e10cb chore: optimize build file make it error sooner on flake8 tests
John Rouillard <rouilj@ieee.org>
parents: 7742
diff changeset
161 flake8 roundup --count --select=E9,F63,F7,F82 --show-source --statistics --extend-exclude talgettext.py
a8891a0e10cb chore: optimize build file make it error sooner on flake8 tests
John Rouillard <rouilj@ieee.org>
parents: 7742
diff changeset
162 # exit-zero treats all errors as warnings.
a8891a0e10cb chore: optimize build file make it error sooner on flake8 tests
John Rouillard <rouilj@ieee.org>
parents: 7742
diff changeset
163 # The GitHub editor is 127 chars wide
a8891a0e10cb chore: optimize build file make it error sooner on flake8 tests
John Rouillard <rouilj@ieee.org>
parents: 7742
diff changeset
164 flake8 roundup --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
a8891a0e10cb chore: optimize build file make it error sooner on flake8 tests
John Rouillard <rouilj@ieee.org>
parents: 7742
diff changeset
165
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
166 # Install the databases
6852
8ffb859b01dd more fixes.
John Rouillard <rouilj@ieee.org>
parents: 6851
diff changeset
167 - name: Install mysql/mariadb
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
168 run: |
6878
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
169 #set -xv
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
170 # mysql is pre-installed and active but this is the install command
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
171 # sudo apt-get install mysql-server mysql-client
6841
3671c0f6352e More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6840
diff changeset
172 # set up mysql database
6870
b040642efae3 mysql sleep and more debug
John Rouillard <rouilj@ieee.org>
parents: 6869
diff changeset
173 sudo sed -i -e '/^\[mysqld\]/,/^\[mysql/s/^#* *max_allowed_packet.*/max_allowed_packet = 500M/' /etc/mysql/mysql.conf.d/mysqld.cnf; sleep 3
6878
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
174 #tail -n +0 /etc/mysql/my.cnf /etc/mysql/mysql.conf.d/mysqld.cnf
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
175 #grep max_allowed /etc/mysql/mysql.conf.d/mysqld.cnf
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
176 #ls /etc/mysql/conf.d/ /etc/mysql/mysql.conf.d/
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
177 #sleep 5
7488
76d4c3d43258 Speed up postgresql by disabiling durability settings.
John Rouillard <rouilj@ieee.org>
parents: 7487
diff changeset
178 # try to improve performance speed by disabling some ACID
76d4c3d43258 Speed up postgresql by disabiling durability settings.
John Rouillard <rouilj@ieee.org>
parents: 7487
diff changeset
179 # settings and change some layout defaults.
7489
f0ff7f790c5c Fix mysql db speedup
John Rouillard <rouilj@ieee.org>
parents: 7488
diff changeset
180 sudo sed -i -e '$a\innodb_flush_log_at_trx_commit = 2' /etc/mysql/mysql.conf.d/mysqld.cnf
f0ff7f790c5c Fix mysql db speedup
John Rouillard <rouilj@ieee.org>
parents: 7488
diff changeset
181 sudo sed -i -e '$a\innodb_file_per_table = OFF' /etc/mysql/mysql.conf.d/mysqld.cnf
f0ff7f790c5c Fix mysql db speedup
John Rouillard <rouilj@ieee.org>
parents: 7488
diff changeset
182 sudo sed -i -e '$a\innodb_doublewrite=OFF' /etc/mysql/mysql.conf.d/mysqld.cnf
f0ff7f790c5c Fix mysql db speedup
John Rouillard <rouilj@ieee.org>
parents: 7488
diff changeset
183 sudo sed -i -e '$a\innodb_fast_shutdown=2' /etc/mysql/mysql.conf.d/mysqld.cnf
f0ff7f790c5c Fix mysql db speedup
John Rouillard <rouilj@ieee.org>
parents: 7488
diff changeset
184 sudo sed -i -e '$a\innodb_log_file_size=1048576' /etc/mysql/mysql.conf.d/mysqld.cnf
f0ff7f790c5c Fix mysql db speedup
John Rouillard <rouilj@ieee.org>
parents: 7488
diff changeset
185 sudo sed -i -e '$a\innodb_flush_method=O_DIRECT' /etc/mysql/mysql.conf.d/mysqld.cnf
f0ff7f790c5c Fix mysql db speedup
John Rouillard <rouilj@ieee.org>
parents: 7488
diff changeset
186 sudo sed -i -e '$a\innodb_log_buffer_size=3M' /etc/mysql/mysql.conf.d/mysqld.cnf
f0ff7f790c5c Fix mysql db speedup
John Rouillard <rouilj@ieee.org>
parents: 7488
diff changeset
187 sudo sed -i -e '$a\innodb_buffer_pool_size=180M' /etc/mysql/mysql.conf.d/mysqld.cnf
7488
76d4c3d43258 Speed up postgresql by disabiling durability settings.
John Rouillard <rouilj@ieee.org>
parents: 7487
diff changeset
188 sleep 3
6878
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
189 sudo service mysql restart
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
190 #sleep 10
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
191 #ps -ef | grep mysqld
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
192 #sudo netstat -anp | grep mysqld
6877
ac2b8a8f5727 fix mysql SQL to create user and grant. set max jobs in parallel
John Rouillard <rouilj@ieee.org>
parents: 6876
diff changeset
193 sudo mysql -u root -proot -e 'CREATE USER "rounduptest"@"localhost" IDENTIFIED WITH mysql_native_password BY "rounduptest"; GRANT ALL on rounduptest.* TO "rounduptest"@"localhost";'
6873
8dab742710b6 null pw??
John Rouillard <rouilj@ieee.org>
parents: 6872
diff changeset
194
8dab742710b6 null pw??
John Rouillard <rouilj@ieee.org>
parents: 6872
diff changeset
195 - name: Install postgres
8dab742710b6 null pw??
John Rouillard <rouilj@ieee.org>
parents: 6872
diff changeset
196 run: |
7417
6b332e3d7c43 try to fix postgres install on ubuntu 20.04 for Python 3.6.
John Rouillard <rouilj@ieee.org>
parents: 7415
diff changeset
197 sudo apt-get update && sudo apt-get install postgresql
7488
76d4c3d43258 Speed up postgresql by disabiling durability settings.
John Rouillard <rouilj@ieee.org>
parents: 7487
diff changeset
198 # Disable fsync, full page writes for speed,
76d4c3d43258 Speed up postgresql by disabiling durability settings.
John Rouillard <rouilj@ieee.org>
parents: 7487
diff changeset
199 # don't care about data durability when testing
6873
8dab742710b6 null pw??
John Rouillard <rouilj@ieee.org>
parents: 6872
diff changeset
200 sudo sed -i -e '$a\fsync = off' /etc/postgresql/*/*/postgresql.conf
7488
76d4c3d43258 Speed up postgresql by disabiling durability settings.
John Rouillard <rouilj@ieee.org>
parents: 7487
diff changeset
201 sudo sed -i -e '$a\full_page_writes = off' /etc/postgresql/*/*/postgresql.conf
76d4c3d43258 Speed up postgresql by disabiling durability settings.
John Rouillard <rouilj@ieee.org>
parents: 7487
diff changeset
202 sudo sed -i -e '$a\synchronous_commit = off' /etc/postgresql/*/*/postgresql.conf
6878
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
203 sudo service postgresql restart; sleep 10
6873
8dab742710b6 null pw??
John Rouillard <rouilj@ieee.org>
parents: 6872
diff changeset
204 # set up postgresql database
8dab742710b6 null pw??
John Rouillard <rouilj@ieee.org>
parents: 6872
diff changeset
205 sudo -u postgres psql -c "CREATE ROLE rounduptest WITH CREATEDB LOGIN PASSWORD 'rounduptest';" -U postgres
7719
3071db43bfb6 feat: issue2550852 - support using a specified PostgreSQL db schema
John Rouillard <rouilj@ieee.org>
parents: 7656
diff changeset
206 sudo -u postgres psql -c "CREATE ROLE rounduptest_schema LOGIN PASSWORD 'rounduptest';" -U postgres
7721
d55427443443 chore: fix ci yaml formatting.
John Rouillard <rouilj@ieee.org>
parents: 7719
diff changeset
207 sudo -u postgres psql -c "CREATE DATABASE rounduptest_schema;" -U postgres
d55427443443 chore: fix ci yaml formatting.
John Rouillard <rouilj@ieee.org>
parents: 7719
diff changeset
208 sudo -u postgres psql -c "GRANT CREATE ON DATABASE rounduptest_schema TO rounduptest_schema;" -U postgres
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
209
6864
7e907d97deb6 redis, gpg, more myql debugging
John Rouillard <rouilj@ieee.org>
parents: 6863
diff changeset
210 - name: install redis
6865
e69b4bfaccd3 pip redis in, fix markdown2 test, install ReST, mysql edit
John Rouillard <rouilj@ieee.org>
parents: 6864
diff changeset
211 run: |
e69b4bfaccd3 pip redis in, fix markdown2 test, install ReST, mysql edit
John Rouillard <rouilj@ieee.org>
parents: 6864
diff changeset
212 sudo apt-get install redis
e69b4bfaccd3 pip redis in, fix markdown2 test, install ReST, mysql edit
John Rouillard <rouilj@ieee.org>
parents: 6864
diff changeset
213 pip install redis
6864
7e907d97deb6 redis, gpg, more myql debugging
John Rouillard <rouilj@ieee.org>
parents: 6863
diff changeset
214
6878
81d811b8d45f set full python version string; other cleanups
John Rouillard <rouilj@ieee.org>
parents: 6877
diff changeset
215 - name: Install python db libraries
7896
dc3bf565794a test(ci): disable psycopg2 install for 3.13
John Rouillard <rouilj@ieee.org>
parents: 7895
diff changeset
216 run: |
dc3bf565794a test(ci): disable psycopg2 install for 3.13
John Rouillard <rouilj@ieee.org>
parents: 7895
diff changeset
217 pip install mysqlclient
8118
c88ad93f0e77 issue2551339 - psycopg2 on 3.13 github runner crashes on import
John Rouillard <rouilj@ieee.org>
parents: 8117
diff changeset
218 pip install psycopg2
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
219
8153
18a391e63202 add new tmate debug point.
John Rouillard <rouilj@ieee.org>
parents: 8152
diff changeset
220 # https://github.com/mxschmitt/action-tmate
18a391e63202 add new tmate debug point.
John Rouillard <rouilj@ieee.org>
parents: 8152
diff changeset
221 # allow remote ssh into the CI container. I need this to debug
18a391e63202 add new tmate debug point.
John Rouillard <rouilj@ieee.org>
parents: 8152
diff changeset
222 # some xfail cases
18a391e63202 add new tmate debug point.
John Rouillard <rouilj@ieee.org>
parents: 8152
diff changeset
223 - name: Setup tmate session
18a391e63202 add new tmate debug point.
John Rouillard <rouilj@ieee.org>
parents: 8152
diff changeset
224 uses: mxschmitt/action-tmate@v3
18a391e63202 add new tmate debug point.
John Rouillard <rouilj@ieee.org>
parents: 8152
diff changeset
225 if: ${{ github.event_name == 'workflow_dispatch' && inputs.debug_enabled }}
18a391e63202 add new tmate debug point.
John Rouillard <rouilj@ieee.org>
parents: 8152
diff changeset
226 timeout-minutes: 10
18a391e63202 add new tmate debug point.
John Rouillard <rouilj@ieee.org>
parents: 8152
diff changeset
227 with:
18a391e63202 add new tmate debug point.
John Rouillard <rouilj@ieee.org>
parents: 8152
diff changeset
228 limit-access-to-actor: true
18a391e63202 add new tmate debug point.
John Rouillard <rouilj@ieee.org>
parents: 8152
diff changeset
229
6865
e69b4bfaccd3 pip redis in, fix markdown2 test, install ReST, mysql edit
John Rouillard <rouilj@ieee.org>
parents: 6864
diff changeset
230 - name: Install auxiliary packages
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
231 run: |
8151
c43b2bd8e203 add gpg package to build - ubuntu-22.04 python 3.13 skips pgp tests.
John Rouillard <rouilj@ieee.org>
parents: 8150
diff changeset
232 sudo apt-get install swig gpg gpgsm libgpgme-dev
6865
e69b4bfaccd3 pip redis in, fix markdown2 test, install ReST, mysql edit
John Rouillard <rouilj@ieee.org>
parents: 6864
diff changeset
233 # pygments for markdown2 to highlight code blocks
7891
3de80157606c fix: revert markdown2 version limitation; markdown2 bug resolved
John Rouillard <rouilj@ieee.org>
parents: 7872
diff changeset
234 pip install markdown2 pygments
6865
e69b4bfaccd3 pip redis in, fix markdown2 test, install ReST, mysql edit
John Rouillard <rouilj@ieee.org>
parents: 6864
diff changeset
235 # docutils for ReStructuredText
8149
ee258d848a59 special case ignore gpg build - 24.04 ubuntu github action migration
John Rouillard <rouilj@ieee.org>
parents: 8148
diff changeset
236 pip install beautifulsoup4 brotli docutils jinja2 \
6866
a3ad2d1a4ea1 spec mistune version; force socket on mysql invocation
John Rouillard <rouilj@ieee.org>
parents: 6865
diff changeset
237 mistune==0.8.4 pyjwt pytz whoosh
8321
71e961941be6 chore(ci): issue2551368 make pip install gpg work for 24.04 ubuntu
John Rouillard <rouilj@ieee.org>
parents: 8309
diff changeset
238 # gpg on PyPi is currently broken with newer OS platform
71e961941be6 chore(ci): issue2551368 make pip install gpg work for 24.04 ubuntu
John Rouillard <rouilj@ieee.org>
parents: 8309
diff changeset
239 # ubuntu 24.04
71e961941be6 chore(ci): issue2551368 make pip install gpg work for 24.04 ubuntu
John Rouillard <rouilj@ieee.org>
parents: 8309
diff changeset
240 # used for newer Python versions. Temporarily use the
71e961941be6 chore(ci): issue2551368 make pip install gpg work for 24.04 ubuntu
John Rouillard <rouilj@ieee.org>
parents: 8309
diff changeset
241 # testing index, which contains a newer version of the
71e961941be6 chore(ci): issue2551368 make pip install gpg work for 24.04 ubuntu
John Rouillard <rouilj@ieee.org>
parents: 8309
diff changeset
242 # bindings on 24.04 or released version for other OS
71e961941be6 chore(ci): issue2551368 make pip install gpg work for 24.04 ubuntu
John Rouillard <rouilj@ieee.org>
parents: 8309
diff changeset
243 # versions. See issue2551368
8323
32a57cde072f chore(ci): issue2551368 use new gpg for all releases.
John Rouillard <rouilj@ieee.org>
parents: 8321
diff changeset
244 if true || grep 24.04 /etc/os-release > /dev/null; then \
8321
71e961941be6 chore(ci): issue2551368 make pip install gpg work for 24.04 ubuntu
John Rouillard <rouilj@ieee.org>
parents: 8309
diff changeset
245 pip install --index-url https://test.pypi.org/simple/ \
71e961941be6 chore(ci): issue2551368 make pip install gpg work for 24.04 ubuntu
John Rouillard <rouilj@ieee.org>
parents: 8309
diff changeset
246 --extra-index-url https://pypi.org/simple --pre gpg; \
71e961941be6 chore(ci): issue2551368 make pip install gpg work for 24.04 ubuntu
John Rouillard <rouilj@ieee.org>
parents: 8309
diff changeset
247 else \
71e961941be6 chore(ci): issue2551368 make pip install gpg work for 24.04 ubuntu
John Rouillard <rouilj@ieee.org>
parents: 8309
diff changeset
248 pip install gpg; \
71e961941be6 chore(ci): issue2551368 make pip install gpg work for 24.04 ubuntu
John Rouillard <rouilj@ieee.org>
parents: 8309
diff changeset
249 fi
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
250
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
251 - name: Install aux packages that need versions differences
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
252 # if zstd fails install, keep going with test, don't abort
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
253 run: |
6842
d8d4600c49eb More fixes to yaml using linter:
John Rouillard <rouilj@ieee.org>
parents: 6841
diff changeset
254 set -xv
6840
8ee17804086e untabify yml file.
John Rouillard <rouilj@ieee.org>
parents: 6839
diff changeset
255 pip install zstd || true
6861
097578791f7b mysql verbose and remove -u
John Rouillard <rouilj@ieee.org>
parents: 6860
diff changeset
256 if [[ "$PYTHON_VERSION" != "2."* ]]; then
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
257 pip install Markdown; fi
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
258
6867
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
259 - name: Install xapian
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
260 run: |
7630
f4e415e56585 build: xapian under 3.12
John Rouillard <rouilj@ieee.org>
parents: 7629
diff changeset
261 set -xv
6867
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
262 sudo apt-get install libxapian-dev
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
263 # Sphinx required to build the xapian python bindings. Use 1.8.5 on
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
264 # older python and newest on newer.
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
265 if [[ $PYTHON_VERSION == "2."* ]]; then pip install sphinx==1.8.5; fi
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
266 if [[ $PYTHON_VERSION == '3.'* ]] ; then pip install sphinx; fi
8158
9fe279eef724 cleanup special case - xapian doesn't build in CI for 3.13 python
John Rouillard <rouilj@ieee.org>
parents: 8153
diff changeset
267 XAPIAN_VER=$(dpkg -l libxapian-dev | tail -n 1 | awk '{print $3}' | cut -d '-' -f 1); echo $XAPIAN_VER;
6867
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
268 cd /tmp
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
269 curl -s -O https://oligarchy.co.uk/xapian/$XAPIAN_VER/xapian-bindings-$XAPIAN_VER.tar.xz
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
270 tar -Jxvf xapian-bindings-$XAPIAN_VER.tar.xz
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
271 cd xapian-bindings-$XAPIAN_VER/
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
272 if [[ $PYTHON_VERSION == "2."* ]]; then ./configure --prefix=$VIRTUAL_ENV --with-python --disable-documentation; fi
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
273 # edit the configure script.
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
274 # distutils.sysconfig.get_config_vars('SO') doesn't work for
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
275 # 3.11 or newer.
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
276 # Change distutils.sysconfig... to just sysconfig and SO
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
277 # to EXT_SUFFIX to get valid value.
7628
88c13f6dbc95 build: try 3.12 xapian build again.
John Rouillard <rouilj@ieee.org>
parents: 7627
diff changeset
278 if [[ $PYTHON_VERSION == "3."* ]]; then \
7630
f4e415e56585 build: xapian under 3.12
John Rouillard <rouilj@ieee.org>
parents: 7629
diff changeset
279 cp configure configure.FCS; \
7628
88c13f6dbc95 build: try 3.12 xapian build again.
John Rouillard <rouilj@ieee.org>
parents: 7627
diff changeset
280 sed -i \
88c13f6dbc95 build: try 3.12 xapian build again.
John Rouillard <rouilj@ieee.org>
parents: 7627
diff changeset
281 -e '/PYTHON3_SO=/s/distutils\.//g' \
88c13f6dbc95 build: try 3.12 xapian build again.
John Rouillard <rouilj@ieee.org>
parents: 7627
diff changeset
282 -e '/PYTHON3_SO=/s/"SO"/"EXT_SUFFIX"/g' \
88c13f6dbc95 build: try 3.12 xapian build again.
John Rouillard <rouilj@ieee.org>
parents: 7627
diff changeset
283 -e '/PYTHON3_CACHE_TAG=/s/imp;print(imp.get_tag())/sys;print(sys.implementation.cache_tag)/' \
88c13f6dbc95 build: try 3.12 xapian build again.
John Rouillard <rouilj@ieee.org>
parents: 7627
diff changeset
284 -e '/PYTHON3_CACHE_OPT1_EXT=/s/imp\.get_tag()/sys.implementation.cache_tag/g' \
7958
fffbaa3f963b test: revert last commit it broke xapian build
John Rouillard <rouilj@ieee.org>
parents: 7957
diff changeset
285 -e '/PYTHON3_CACHE_OPT1_EXT=/s/imp\b/importlib/g' \
7628
88c13f6dbc95 build: try 3.12 xapian build again.
John Rouillard <rouilj@ieee.org>
parents: 7627
diff changeset
286 configure; \
7632
0334d371ea11 build: prevent diff from exiting shell script.
John Rouillard <rouilj@ieee.org>
parents: 7631
diff changeset
287 diff -u configure.FCS configure || true; \
7628
88c13f6dbc95 build: try 3.12 xapian build again.
John Rouillard <rouilj@ieee.org>
parents: 7627
diff changeset
288 ./configure --prefix=$VIRTUAL_ENV --with-python3 --disable-documentation; \
7629
bad8affdcba7 build: fix indent in yaml.
John Rouillard <rouilj@ieee.org>
parents: 7628
diff changeset
289 fi
8158
9fe279eef724 cleanup special case - xapian doesn't build in CI for 3.13 python
John Rouillard <rouilj@ieee.org>
parents: 8153
diff changeset
290 make && sudo make install
6867
25487b21f812 add xapian build/install
John Rouillard <rouilj@ieee.org>
parents: 6866
diff changeset
291
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
292 - name: Test build roundup and install locale so lang tests work.
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
293 run: |
6855
30815d9bd95c fix tab
John Rouillard <rouilj@ieee.org>
parents: 6854
diff changeset
294 sudo apt-get install gettext
6854
4f115313e262 install gettext for locale
John Rouillard <rouilj@ieee.org>
parents: 6853
diff changeset
295 python setup.py build
6840
8ee17804086e untabify yml file.
John Rouillard <rouilj@ieee.org>
parents: 6839
diff changeset
296 (cd locale; make local_install; ls -lR locale/de/LC_MESSAGES)
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
297
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
298 # Run the tests using pytest with test files in tests directory.
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
299 - name: Run tests
6858
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
300 run: |
6861
097578791f7b mysql verbose and remove -u
John Rouillard <rouilj@ieee.org>
parents: 6860
diff changeset
301 if [[ "$PYTHON_VERSION" != "2."* ]]; then
6858
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
302 pytest -r a \
7169
5cbe5f2a636b Add --durations=20 for detecting perf changes.
John Rouillard <rouilj@ieee.org>
parents: 7148
diff changeset
303 --durations=20 \
6858
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
304 -W default \
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
305 -W "ignore:SelectableGroups:DeprecationWarning" \
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
306 -W "ignore:the imp module:DeprecationWarning:gpg.gpgme:15" \
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
307 -W "ignore:'U' mode::docutils.io" \
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
308 -W "ignore:unclosed:ResourceWarning:roundup.roundup.demo" \
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
309 -W "ignore:unclosed file:ResourceWarning:enum" \
6952
4eea63155cff enable all tests, make test job failure stil run final job.
John Rouillard <rouilj@ieee.org>
parents: 6951
diff changeset
310 -v test/ --cov=roundup
6951
31c56e8dfefb 3.6 python doesn't support lcov format either.
John Rouillard <rouilj@ieee.org>
parents: 6950
diff changeset
311 coverage lcov
6858
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
312 else
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
313 # python2 case
7171
4e3abaa16296 add duration reporting for python2 as well.
John Rouillard <rouilj@ieee.org>
parents: 7169
diff changeset
314 pytest -v -r a --durations=20 test/ --cov=roundup
6858
John Rouillard <rouilj@ieee.org>
parents: 6857
diff changeset
315 fi
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
316
7742
b7a8d9664938 doc: fix build_docs test on travis and add to github ci
John Rouillard <rouilj@ieee.org>
parents: 7729
diff changeset
317 - name: Build docs
b7a8d9664938 doc: fix build_docs test on travis and add to github ci
John Rouillard <rouilj@ieee.org>
parents: 7729
diff changeset
318 run: python ./setup.py build_doc
b7a8d9664938 doc: fix build_docs test on travis and add to github ci
John Rouillard <rouilj@ieee.org>
parents: 7729
diff changeset
319
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
320 - name: Upload coverage to Codecov
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
321 # see: https://github.com/codecov/codecov-action#usage
7729
edd93fa4a5ec chore: update actions in gihub workflows.
John Rouillard <rouilj@ieee.org>
parents: 7721
diff changeset
322 uses: codecov/codecov-action@4fe8c5f003fae66aa5ebb77cfd3e7bfbbda0b6b0 # v3.1.5
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
323 with:
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
324 verbose: true
6881
ae3d98c1fb69 fix tab.
John Rouillard <rouilj@ieee.org>
parents: 6880
diff changeset
325 token: ${{ secrets.CODECOV_TOKEN }}
6839
6e1a82c3addb Add GitHub Actions test workflow.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
326
6945
7eb0e1cffe54 indentation fix.
John Rouillard <rouilj@ieee.org>
parents: 6944
diff changeset
327 - name: Upload coverage to Coveralls
6951
31c56e8dfefb 3.6 python doesn't support lcov format either.
John Rouillard <rouilj@ieee.org>
parents: 6950
diff changeset
328 # python 2.7 and 3.6 versions of coverage can't produce lcov files.
31c56e8dfefb 3.6 python doesn't support lcov format either.
John Rouillard <rouilj@ieee.org>
parents: 6950
diff changeset
329 if: matrix.python-version != '2.7' && matrix.python-version != '3.6'
7729
edd93fa4a5ec chore: update actions in gihub workflows.
John Rouillard <rouilj@ieee.org>
parents: 7721
diff changeset
330 uses: coverallsapp/github-action@3dfc5567390f6fa9267c0ee9c251e4c8c3f18949 # v2.2.3
6945
7eb0e1cffe54 indentation fix.
John Rouillard <rouilj@ieee.org>
parents: 6944
diff changeset
331 with:
7eb0e1cffe54 indentation fix.
John Rouillard <rouilj@ieee.org>
parents: 6944
diff changeset
332 github-token: ${{ secrets.GITHUB_TOKEN }}
6948
078073f7d207 coveralls for 3.x only, add paralle unique build key
John Rouillard <rouilj@ieee.org>
parents: 6947
diff changeset
333 path-to-lcov: coverage.lcov
078073f7d207 coveralls for 3.x only, add paralle unique build key
John Rouillard <rouilj@ieee.org>
parents: 6947
diff changeset
334 parallel: run-{{ matrix.python-version }}-{{ matrix.os }}
6943
3b081f5c8fc2 add coveralls as codecov is severly broken.
John Rouillard <rouilj@ieee.org>
parents: 6924
diff changeset
335
6893
d668e2800147 more debug
John Rouillard <rouilj@ieee.org>
parents: 6892
diff changeset
336 #- name: test docker build current version ubuntu-latest
6904
92b64fd3341f cleanup, comments ready for production.
John Rouillard <rouilj@ieee.org>
parents: 6903
diff changeset
337 # if: matrix.python-version == '3.10' && matrix.os == 'ubuntu-latest'
6880
222e84d12bed schedule build 7PM on first of month; commented out docker tst
John Rouillard <rouilj@ieee.org>
parents: 6879
diff changeset
338 # run: |
222e84d12bed schedule build 7PM on first of month; commented out docker tst
John Rouillard <rouilj@ieee.org>
parents: 6879
diff changeset
339 # docker build -t roundup-app-dev -f scripts/Docker/Dockerfile .
222e84d12bed schedule build 7PM on first of month; commented out docker tst
John Rouillard <rouilj@ieee.org>
parents: 6879
diff changeset
340 # mkdir tracker; chmod 777 tracker
222e84d12bed schedule build 7PM on first of month; commented out docker tst
John Rouillard <rouilj@ieee.org>
parents: 6879
diff changeset
341 # docker run -d --rm -p 9017:8080 \
222e84d12bed schedule build 7PM on first of month; commented out docker tst
John Rouillard <rouilj@ieee.org>
parents: 6879
diff changeset
342 # -v $PWD/tracker:/usr/src/app/tracker \
222e84d12bed schedule build 7PM on first of month; commented out docker tst
John Rouillard <rouilj@ieee.org>
parents: 6879
diff changeset
343 # roundup-app-dev:latest demo
222e84d12bed schedule build 7PM on first of month; commented out docker tst
John Rouillard <rouilj@ieee.org>
parents: 6879
diff changeset
344 # expect 200
222e84d12bed schedule build 7PM on first of month; commented out docker tst
John Rouillard <rouilj@ieee.org>
parents: 6879
diff changeset
345 # curl --fail http://localhost:9017/demo/ > /dev/null
222e84d12bed schedule build 7PM on first of month; commented out docker tst
John Rouillard <rouilj@ieee.org>
parents: 6879
diff changeset
346
222e84d12bed schedule build 7PM on first of month; commented out docker tst
John Rouillard <rouilj@ieee.org>
parents: 6879
diff changeset
347 #- name: test docker build released pip version
222e84d12bed schedule build 7PM on first of month; commented out docker tst
John Rouillard <rouilj@ieee.org>
parents: 6879
diff changeset
348 # run: |
222e84d12bed schedule build 7PM on first of month; commented out docker tst
John Rouillard <rouilj@ieee.org>
parents: 6879
diff changeset
349 # docker build -t roundup-app-rel --build-arg="source=pypi" \
222e84d12bed schedule build 7PM on first of month; commented out docker tst
John Rouillard <rouilj@ieee.org>
parents: 6879
diff changeset
350 # -f scripts/Docker/Dockerfile .
6943
3b081f5c8fc2 add coveralls as codecov is severly broken.
John Rouillard <rouilj@ieee.org>
parents: 6924
diff changeset
351
3b081f5c8fc2 add coveralls as codecov is severly broken.
John Rouillard <rouilj@ieee.org>
parents: 6924
diff changeset
352
7871
30fcdf60da44 test: fix benchmark.py and set up to run under github if requested
John Rouillard <rouilj@ieee.org>
parents: 7857
diff changeset
353 - name: run benchmarks
30fcdf60da44 test: fix benchmark.py and set up to run under github if requested
John Rouillard <rouilj@ieee.org>
parents: 7857
diff changeset
354 if: "contains(github.event.head_commit.message, 'benchmark')"
7872
163d2c60fdf3 test: benchmark no progress when INCI defined; set path; signal handling
John Rouillard <rouilj@ieee.org>
parents: 7871
diff changeset
355 run: INCI=1 python test/benchmark.py
7871
30fcdf60da44 test: fix benchmark.py and set up to run under github if requested
John Rouillard <rouilj@ieee.org>
parents: 7857
diff changeset
356
7390
399093a75afa Run coveralls push only if all builds succeed.
John Rouillard <rouilj@ieee.org>
parents: 7274
diff changeset
357 # in parallel build coveralls requires a finish step
6945
7eb0e1cffe54 indentation fix.
John Rouillard <rouilj@ieee.org>
parents: 6944
diff changeset
358 finish:
7eb0e1cffe54 indentation fix.
John Rouillard <rouilj@ieee.org>
parents: 6944
diff changeset
359 needs: test
7eb0e1cffe54 indentation fix.
John Rouillard <rouilj@ieee.org>
parents: 6944
diff changeset
360 runs-on: ubuntu-latest
7194
8dc5b3739367 Prevent github actions from running if commit includes 'no-github-ci'
John Rouillard <rouilj@ieee.org>
parents: 7187
diff changeset
361
8dc5b3739367 Prevent github actions from running if commit includes 'no-github-ci'
John Rouillard <rouilj@ieee.org>
parents: 7187
diff changeset
362 if: "!contains(github.event.head_commit.message, 'no-github-ci')"
8dc5b3739367 Prevent github actions from running if commit includes 'no-github-ci'
John Rouillard <rouilj@ieee.org>
parents: 7187
diff changeset
363
6945
7eb0e1cffe54 indentation fix.
John Rouillard <rouilj@ieee.org>
parents: 6944
diff changeset
364 steps:
7eb0e1cffe54 indentation fix.
John Rouillard <rouilj@ieee.org>
parents: 6944
diff changeset
365 - name: Coveralls Finished
7729
edd93fa4a5ec chore: update actions in gihub workflows.
John Rouillard <rouilj@ieee.org>
parents: 7721
diff changeset
366 uses: coverallsapp/github-action@3dfc5567390f6fa9267c0ee9c251e4c8c3f18949 # v2.2.3
6945
7eb0e1cffe54 indentation fix.
John Rouillard <rouilj@ieee.org>
parents: 6944
diff changeset
367 with:
7eb0e1cffe54 indentation fix.
John Rouillard <rouilj@ieee.org>
parents: 6944
diff changeset
368 github-token: ${{ secrets.github_token }}
7eb0e1cffe54 indentation fix.
John Rouillard <rouilj@ieee.org>
parents: 6944
diff changeset
369 parallel-finished: true
Roundup Issue Tracker: http://roundup-tracker.org/