Skip to content

Commit 3dac0fa

Browse files
crazyscientistcrobinso
authored andcommitted
Run functional RO tests in GitHub actions
* Run MariaDB and Bugzilla in service containers * Populate the DB with a defined dump * Include all files to build the Bugzilla image and prepare the environment * Implemented integration tests in new test module * Added a new fixture for request mocking
1 parent 178fb6f commit 3dac0fa

14 files changed

Lines changed: 2750 additions & 0 deletions

File tree

.github/workflows/build.yml

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,49 @@ jobs:
6666
with:
6767
token: ${{ secrets.CODECOV_TOKEN }}
6868

69+
# Run functional tests
70+
integrationRO:
71+
runs-on: ubuntu-latest
72+
services:
73+
mariadb:
74+
image: mariadb:latest
75+
env:
76+
MARIADB_USER: bugs
77+
MARIADB_DATABASE: bugs
78+
MARIADB_PASSWORD: secret
79+
MARIADB_ROOT_PASSWORD: supersecret
80+
ports:
81+
- 3306:3306
82+
bugzilla:
83+
image: ghcr.io/crazyscientist/bugzilla:test
84+
ports:
85+
- 80:80
86+
strategy:
87+
matrix:
88+
python-version: ["3.x"]
89+
steps:
90+
- uses: actions/checkout@v3
91+
- name: Install MariaDB utils
92+
run: sudo apt install --no-install-recommends -q -y mariadb-client
93+
- name: Restore DB dump
94+
run: mariadb -h 127.0.0.1 -P 3306 --password=secret -u bugs bugs < tests/services/bugs.sql
95+
- name: Store API key
96+
run: |
97+
mkdir -p ~/.config/python-bugzilla/
98+
cp tests/services/bugzillarc ~/.config/python-bugzilla/
99+
- name: Set up Python ${{ matrix.python-version }}
100+
uses: actions/setup-python@v4
101+
with:
102+
python-version: ${{ matrix.python-version }}
103+
- name: Install dependencies
104+
run: |
105+
python -m pip install --upgrade pip
106+
pip install pytest pytest-cov
107+
pip install -r requirements.txt -r test-requirements.txt
108+
- name: Test with pytest
109+
run: pytest --ro-integration
110+
env:
111+
BUGZILLA_URL: http://localhost
69112

70113
# Build and install on Windows
71114
windows:

test-requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,4 @@
22
pytest
33
pylint<3.1
44
pycodestyle<2.12
5+
responses

tests/conftest.py

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,10 @@
44
import locale
55
import logging
66
import os
7+
import re
78

89
import pytest
10+
import responses
911

1012
import tests
1113
import tests.utils
@@ -17,6 +19,8 @@
1719
# https://docs.pytest.org/en/latest/writing_plugins.html
1820

1921
def pytest_addoption(parser):
22+
parser.addoption("--ro-integration", action="store_true", default=False,
23+
help="Run readonly tests against local Bugzilla instance.")
2024
parser.addoption("--ro-functional", action="store_true", default=False,
2125
help=("Run readonly functional tests against actual "
2226
"bugzilla instances. This will be very slow."))
@@ -40,11 +44,17 @@ def pytest_addoption(parser):
4044

4145
def pytest_ignore_collect(path, config):
4246
has_ro = config.getoption("--ro-functional")
47+
has_ro_i = config.getoption("--ro-integration")
4348
has_rw = config.getoption("--rw-functional")
4449

4550
base = os.path.basename(str(path))
4651
is_ro = base == "test_ro_functional.py"
52+
is_ro_i = "tests/integration/ro" in str(path)
4753
is_rw = base == "test_rw_functional.py"
54+
55+
if is_ro_i and not has_ro_i:
56+
return True
57+
4858
if is_ro and not has_ro:
4959
return True
5060
if is_rw and not has_rw:
@@ -107,3 +117,47 @@ def run_cli(capsys, monkeypatch):
107117
def _do_run(*args, **kwargs):
108118
return tests.utils.do_run_cli(capsys, monkeypatch, *args, **kwargs)
109119
return _do_run
120+
121+
122+
@pytest.fixture
123+
def mocked_responses():
124+
"""
125+
Mock responses
126+
127+
* Quickly return error responses
128+
* Pass through requests to live instances
129+
* Provide an incorrect XMLRPC response
130+
"""
131+
passthrough = ()
132+
status_pattern = re.compile(r"https://httpstat.us/(?P<status>\d+).*")
133+
134+
def status_callback(request):
135+
match = status_pattern.match(request.url)
136+
status_code = 400
137+
if match:
138+
status_code = int(match.group("status"))
139+
140+
return status_code, {}, "<html><body><h1>Lorem ipsum</h1></body></html>"
141+
142+
test_url = os.getenv("BUGZILLA_URL")
143+
if test_url:
144+
passthrough += (test_url, )
145+
with responses.RequestsMock(passthru_prefixes=passthrough,
146+
assert_all_requests_are_fired=False) as mock:
147+
mock.add_callback(
148+
method=responses.GET,
149+
url=status_pattern,
150+
callback=status_callback
151+
)
152+
mock.add_callback(
153+
method=responses.POST,
154+
url=status_pattern,
155+
callback=status_callback
156+
)
157+
mock.add(
158+
method=responses.POST,
159+
url="https://example.com/#xmlrpc",
160+
status=200,
161+
body="This is no XML"
162+
)
163+
yield mock

tests/integration/__init__.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
import os
2+
3+
4+
TEST_URL = os.getenv("BUGZILLA_URL", "http://localhost")
5+
TEST_OWNER = "andreas@hasenkopf.xyz"
6+
TEST_PRODUCTS = {"Red Hat Enterprise Linux 9",
7+
"SUSE Linux Enterprise Server 15 SP6",
8+
"TestProduct"}
9+
TEST_SUSE_COMPONENTS = {"Containers", "Kernel"}

tests/integration/ro_api_test.py

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
# Ignoring pytest-related warnings:
2+
# pylint: disable=redefined-outer-name,unused-argument
3+
import pytest
4+
5+
from bugzilla import BugzillaError
6+
7+
from ..utils import open_bz
8+
from . import TEST_URL, TEST_PRODUCTS, TEST_SUSE_COMPONENTS, TEST_OWNER
9+
10+
11+
def test_rest_xmlrpc_detection(mocked_responses):
12+
# The default: use XMLRPC
13+
bz = open_bz(url=TEST_URL)
14+
assert bz.is_xmlrpc()
15+
assert "/xmlrpc.cgi" in bz.url
16+
17+
# See /rest in the URL, so use REST
18+
bz = open_bz(url=TEST_URL + "/rest")
19+
assert bz.is_rest()
20+
with pytest.raises(BugzillaError) as e:
21+
dummy = bz._proxy # pylint: disable=protected-access
22+
assert "raw XMLRPC access is not provided" in str(e)
23+
24+
# See /xmlrpc.cgi in the URL, so use XMLRPC
25+
bz = open_bz(url=TEST_URL + "/xmlrpc.cgi")
26+
assert "/xmlrpc.cgi" in bz.url
27+
assert bz.is_xmlrpc()
28+
assert bz._proxy # pylint: disable=protected-access
29+
30+
31+
def test_apikey_error_scraping(mocked_responses):
32+
# Ensure the API key does not leak into any requests exceptions
33+
fakekey = "FOOBARMYKEY"
34+
with pytest.raises(Exception) as e:
35+
open_bz("https://httpstat.us/400&foo",
36+
force_xmlrpc=True, api_key=fakekey)
37+
assert "Client Error" in str(e.value)
38+
assert fakekey not in str(e.value)
39+
40+
with pytest.raises(Exception) as e:
41+
open_bz("https://httpstat.us/400&foo",
42+
force_rest=True, api_key=fakekey)
43+
assert "Client Error" in str(e.value)
44+
assert fakekey not in str(e.value)
45+
46+
47+
def test_xmlrpc_bad_url(mocked_responses):
48+
with pytest.raises(BugzillaError) as e:
49+
open_bz(url="https://example.com/#xmlrpc", force_xmlrpc=True)
50+
assert "URL may not be an XMLRPC URL" in str(e)
51+
52+
53+
def test_get_products(mocked_responses, backends):
54+
bz = open_bz(url=TEST_URL, **backends)
55+
56+
assert len(bz.products) == 3
57+
assert {p["name"] for p in bz.products} == TEST_PRODUCTS
58+
59+
rhel = next(p for p in bz.products if p["id"] == 2)
60+
assert {v["name"] for v in rhel["versions"]} == {"9.0", "9.1", "unspecified"}
61+
62+
63+
def test_get_components(mocked_responses, backends):
64+
bz = open_bz(url=TEST_URL, **backends)
65+
components = bz.getcomponents(product="SUSE Linux Enterprise Server 15 SP6")
66+
assert len(components) == 2
67+
assert set(components) == TEST_SUSE_COMPONENTS
68+
69+
70+
def test_get_component_detail(mocked_responses, backends):
71+
bz = open_bz(url=TEST_URL, **backends)
72+
component = bz.getcomponentdetails(product="Red Hat Enterprise Linux 9",
73+
component="python-bugzilla")
74+
assert component["id"] == 2
75+
assert component["default_assigned_to"] == TEST_OWNER
76+
77+
78+
def test_query(mocked_responses, backends):
79+
bz = open_bz(url=TEST_URL, **backends)
80+
query = bz.build_query(product="Red Hat Enterprise Linux 9", component="python-bugzilla")
81+
bugs = bz.query(query=query)
82+
83+
assert len(bugs) == 1
84+
assert bugs[0].id == 2
85+
assert bugs[0].summary == "Expect the Spanish inquisition"
86+
87+
bz = open_bz(url=TEST_URL, **backends)
88+
query = bz.build_query(product="SUSE Linux Enterprise Server 15 SP6")
89+
bugs = bz.query(query=query)
90+
91+
assert len(bugs) == 1
92+
assert bugs[0].id == 1
93+
assert bugs[0].whiteboard == "AV:N/AC:L/PR:H/UI:N/S:U/C:L/I:N/A:L"
94+
95+
96+
def test_get_bug_alias(mocked_responses, backends):
97+
bz = open_bz(url=TEST_URL, **backends)
98+
bug = bz.getbug("FOO-1")
99+
100+
assert bug.id == 1
101+
assert bug.summary == "ZeroDivisionError in function foo_bar()"

tests/integration/ro_cli_test.py

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# Ignoring pytest-related warnings:
2+
# pylint: disable=unused-argument
3+
from ..utils import open_bz
4+
from . import TEST_URL, TEST_PRODUCTS, TEST_SUSE_COMPONENTS, TEST_OWNER
5+
6+
7+
def test_get_products(mocked_responses, run_cli, backends):
8+
bz = open_bz(url=TEST_URL, **backends)
9+
out = run_cli("bugzilla info --products", bzinstance=bz)
10+
assert len(out.strip().split("\n")) == 3
11+
12+
for product in TEST_PRODUCTS:
13+
assert product in out
14+
15+
16+
def test_get_components(mocked_responses, run_cli, backends):
17+
bz = open_bz(url=TEST_URL, **backends)
18+
out = run_cli("bugzilla info --components 'SUSE Linux Enterprise Server 15 SP6'", bzinstance=bz)
19+
assert len(out.strip().split("\n")) == 2
20+
for comp in TEST_SUSE_COMPONENTS:
21+
assert comp in out
22+
23+
24+
def test_get_component_owners(mocked_responses, run_cli, backends):
25+
bz = open_bz(url=TEST_URL, **backends)
26+
out = run_cli("bugzilla info --component_owners 'SUSE Linux Enterprise Server 15 SP6'",
27+
bzinstance=bz)
28+
assert TEST_OWNER in out
29+
30+
31+
def test_get_versions(mocked_responses, run_cli, backends):
32+
bz = open_bz(url=TEST_URL, **backends)
33+
out = run_cli("bugzilla info --versions 'Red Hat Enterprise Linux 9'", bzinstance=bz)
34+
versions = set(out.strip().split("\n"))
35+
36+
assert versions == {"unspecified", "9.0", "9.1"}
37+
38+
39+
def test_query(mocked_responses, run_cli, backends):
40+
bz = open_bz(url=TEST_URL, **backends)
41+
out = run_cli("bugzilla query --product 'Red Hat Enterprise Linux 9' "
42+
"--component 'python-bugzilla'", bzinstance=bz)
43+
lines = out.strip().splitlines()
44+
45+
assert len(lines) == 1
46+
assert lines[0].startswith("#2")
47+
assert "Expect the Spanish inquisition" in lines[0]

tests/services/Dockerfile

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
FROM ubuntu:22.04
2+
LABEL description="Bugzilla image for testing purposes"
3+
ARG DEBIAN_FRONTEND=noninteractive
4+
ENV TZ="Etc/UTC"
5+
RUN apt update && \
6+
apt install --no-install-recommends -q -y \
7+
tzdata wget apache2 libcgi-pm-perl libdatetime-perl libdatetime-timezone-perl libdbi-perl \
8+
libdbix-connector-perl libdigest-sha-perl libemail-address-perl libemail-mime-perl \
9+
libemail-sender-perl libjson-xs-perl liblist-moreutils-perl libmath-random-isaac-perl \
10+
libtemplate-perl libtimedate-perl liburi-perl libmariadb-dev-compat libdbd-mysql-perl \
11+
libxmlrpc-lite-perl libsoap-lite-perl libapache2-mod-perl2 libtest-taint-perl \
12+
libjson-rpc-perl && \
13+
apt clean
14+
RUN mkdir -p /var/www/webapps && \
15+
wget https://ftp.mozilla.org/pub/mozilla.org/webtools/bugzilla-5.0.6.tar.gz \
16+
-O /tmp/bugzilla-5.0.6.tar.gz&& \
17+
tar xvzf /tmp/bugzilla-5.0.6.tar.gz && \
18+
rm /tmp/bugzilla-5.0.6.tar.gz && \
19+
mv /bugzilla-5.0.6/ /var/www/webapps/bugzilla/ && \
20+
mkdir /var/www/webapps/bugzilla/data/
21+
COPY bugzilla.conf /etc/apache2/sites-available/
22+
COPY localconfig /var/www/webapps/bugzilla/
23+
COPY params.json /var/www/webapps/bugzilla/data/
24+
RUN a2dissite 000-default && \
25+
a2ensite bugzilla && \
26+
a2enmod cgi headers expires rewrite perl && \
27+
/var/www/webapps/bugzilla/checksetup.pl
28+
CMD apachectl -D FOREGROUND

tests/services/README.md

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# Working with the containerized Bugzilla instance
2+
3+
This document describes the steps for building a Bugzilla container image that can be used in the
4+
GitHub Actions as a service and generating a database dump.
5+
6+
In the following examples, the use of `docker` is assumed. Commands for `podman` should be
7+
identical.
8+
9+
## Build
10+
11+
```shell
12+
$ docker network create --driver bridge local-bridge
13+
$ docker run --rm -itd \
14+
--env MARIADB_USER=bugs \
15+
--env MARIADB_DATABASE=bugs \
16+
--env MARIADB_PASSWORD=secret \
17+
--env MARIADB_ROOT_PASSWORD=supersecret \
18+
-p 3306:3306 \
19+
--network local-bridge \
20+
--name mariadb \
21+
mariadb:latest
22+
$ mariadb -u bugs -h 127.0.0.1 -P 3306 --password=secret bugs < bugs.sql
23+
$ docker build --network local-bridge . -t ghcr.io/crazyscientist/bugzilla:test
24+
```
25+
26+
For those, who can spot the _chicken and egg problem_: The first version of `bugs.sql` was
27+
created after running the Bugzilla installer inside the container.
28+
29+
## Usage
30+
31+
Once built, you can follow the above instructions; instead of building
32+
the image, you can run it:
33+
34+
```shell
35+
docker run --rm -itd \
36+
-p 8000:80 \
37+
--network local-bridge \
38+
ghcr.io/crazyscientist/bugzilla:test
39+
```
40+
41+
## Test data
42+
43+
The test data used by the Bugzilla service in the integration test suite is stored in `bugs.sql`.
44+
45+
One can edit this file manually or follow the above instructions to start both a MariaDB and
46+
Bugzilla container and edit the data in Bugzilla. Once done, one needs to dump the changed data into
47+
the file again:
48+
49+
```shell
50+
$ mariadb-dump -u bugs -h 127.0.0.1 -P 3306 --password=secret bugs > bugs.qql
51+
```
52+
53+
## Testing
54+
And now, you can run the integration tests against this instance:
55+
56+
```shell
57+
BUGZILLA_URL=http://localhost:8000 pytest --ro-integration
58+
```

0 commit comments

Comments
 (0)