Skip to content

Commit d5cb802

Browse files
mriggerclaude
mrigger
and
claude
committed
Make CI run the pipeline's actual dependencies, on a Ruby the site supports
Two failures on the first run of workflows that had never run anywhere, and both were real rather than incidental to CI. pypdf was missing. It is not an optional extra: intake reads a supplied PDF's title and first page with it, fulltext extracts paper text with it, and the artifact scanner finds links with it. Every one of those wraps the import in a try/except and returns "nothing found" when it fails, which is right at runtime and dangerous in a dependency list -- the weekly job installs from requirements.txt, so it would have run green while quietly extracting no paper text at all. The test that builds a PDF fixture was the only caller that said so out loud. It is now listed, and that test skips rather than fails without it, because a checkout without pypdf is a supported state. The hand-written `pip install jsonschema` is what let the two drift, so CI now installs from requirements.txt like the other workflow already did. Ruby 3.2 cannot build this site. The github-pages gem pins liquid 4.0.3, which calls Object#tainted?, removed in Ruby 3.2 -- so the build died in a 2023 blog post that has nothing to do with any of this. Its jekyll 3.9.0 also predates the Psych 4 that ships with Ruby 3.1. Ruby 3.0 is the newest that satisfies both, and is effectively what GitHub Pages builds with; the runner moves to 22.04 because setup-ruby ships no 3.0 for 24.04. Nothing about the site changes -- the pin was simply newer than the gems it had to run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EvUdZCeqsafccQ8rt7jt3N
1 parent 5930594 commit d5cb802

4 files changed

Lines changed: 37 additions & 11 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,9 @@ jobs:
2020
- uses: actions/setup-python@v5
2121
with:
2222
python-version: "3.11"
23-
- run: pip install jsonschema
23+
# From requirements.txt rather than a hand-kept list: the two drifted
24+
# once already, and CI is where that surfaces as a mystery.
25+
- run: pip install -r requirements.txt
2426
- name: Validate the impact dataset
2527
run: python -m tools.impact.run validate
2628
- name: Check statistics and charts match the records
@@ -29,12 +31,18 @@ jobs:
2931
run: python -m unittest discover -s tools/impact/tests -t . -v
3032

3133
site:
32-
runs-on: ubuntu-latest
34+
runs-on: ubuntu-22.04
3335
steps:
3436
- uses: actions/checkout@v4
3537
- uses: ruby/setup-ruby@v1
3638
with:
37-
ruby-version: "3.2"
39+
# Pinned low on purpose. The site is built with the github-pages
40+
# gem, which pins liquid 4.0.3 -- and liquid 4.0.3 calls
41+
# Object#tainted?, removed in Ruby 3.2. Its jekyll 3.9.0 also predates
42+
# the Psych 4 shipped with Ruby 3.1. Ruby 3.0 is the newest that
43+
# satisfies both, so it is what GitHub Pages itself effectively builds
44+
# with; raising it here breaks the build without touching the site.
45+
ruby-version: "3.0"
3846
bundler-cache: true
3947
- name: Build the site
4048
run: bundle exec jekyll build --trace

‎.github/workflows/impact.yml‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,9 @@ concurrency:
3838

3939
jobs:
4040
collect:
41-
runs-on: ubuntu-latest
41+
# 22.04 because this job also builds the site, and setup-ruby ships no
42+
# Ruby 3.0 for 24.04 -- see the pin below for why 3.0 is required.
43+
runs-on: ubuntu-22.04
4244
steps:
4345
- uses: actions/checkout@v4
4446

@@ -123,7 +125,13 @@ jobs:
123125
if: steps.changes.outputs.dirty == 'true'
124126
uses: ruby/setup-ruby@v1
125127
with:
126-
ruby-version: "3.2"
128+
# Pinned low on purpose. The site is built with the github-pages
129+
# gem, which pins liquid 4.0.3 -- and liquid 4.0.3 calls
130+
# Object#tainted?, removed in Ruby 3.2. Its jekyll 3.9.0 also predates
131+
# the Psych 4 shipped with Ruby 3.1. Ruby 3.0 is the newest that
132+
# satisfies both, so it is what GitHub Pages itself effectively builds
133+
# with; raising it here breaks the build without touching the site.
134+
ruby-version: "3.0"
127135
bundler-cache: true
128136

129137
- name: Build the website

‎requirements.txt‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Dependencies for the impact collection pipeline (tools/impact).
2-
# Deliberately minimal: the collectors use urllib from the standard library,
3-
# so only schema validation and the optional classifier need a package.
2+
# Deliberately minimal: the collectors use urllib from the standard library, so
3+
# only schema validation, reading PDFs, and the optional classifier need one.
44
jsonschema>=4.0
5-
anthropic>=0.40 # optional at runtime; without a key the classifier is skipped
5+
pypdf>=3.0 # reads supplied and fetched PDFs; every use degrades to "no
6+
# full text" without it, so a missing copy is silent
7+
anthropic>=0.40 # optional at runtime; without a key the classifier is skipped

‎tools/impact/tests/test_pipeline.py‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1804,9 +1804,17 @@ class IntakeReportingTest(unittest.TestCase):
18041804
"""
18051805

18061806
def _pdf(self, tmp, name, text):
1807-
"""A one-page PDF whose page text is ``text``."""
1808-
import pypdf
1809-
from pypdf.generic import DecodedStreamObject, NameObject
1807+
"""A one-page PDF whose page text is ``text``.
1808+
1809+
Skips rather than fails without pypdf. Every production caller treats
1810+
the package as optional and degrades to "no full text", so a checkout
1811+
without it is a supported state, not a broken one.
1812+
"""
1813+
try:
1814+
import pypdf
1815+
from pypdf.generic import DecodedStreamObject, NameObject
1816+
except ImportError:
1817+
self.skipTest("pypdf is not installed; PDF reading is optional")
18101818
writer = pypdf.PdfWriter()
18111819
writer.add_blank_page(width=612, height=792)
18121820
stream = DecodedStreamObject()

0 commit comments

Comments
 (0)