Skip to content

Pin Bouncy Castle versions in JRuby's build to avoid range resolution in CI #10038

Open

Description

@zev

Some master CI jobs fail in "JRuby Lib Setup" with:

[ERROR] No versions available for org.bouncycastle:bcprov-jdk18on:jar:[1.80,1.86] within specified range
[ERROR] No versions available for org.bouncycastle:bctls-jdk18on:jar:[1.80,1.86.1] within specified range

How often: I went through every failed job in the 171 "JRuby CI" runs on master from Sep 29 to Nov 11. jruby-openssl started using ranges on master on Sep 26, with 0.19.0.

  • 11 jobs in 9 runs failed with this error. The first was Oct 20, the most recent Nov 11: run 42930321295, language specs (FreeBSD).
  • In 7 of those 9 runs, every failing job failed on dependency resolution. No test failed, so the range error (sometimes along with other download errors) is the only reason those runs are red.
  • It's part of a bigger pattern: 15 of the 67 failed master runs in that window failed only on Maven/network resolution.
  • In 4 of the 11 jobs, Maven Central was also returning 429 Too Many Requests.

Example runs: 41114666542, 41552658149, 41764923695, 42930321295.

The ranges come from jruby-openssl's gemspec requirements. To resolve a range, Maven has to download maven-metadata.xml for each artifact. It does this for every artifact on every repository (central, rubygems.org via mavengem, and Sonatype snapshots), even when the jars are already in the restored ~/.m2 cache. If a fetch fails, the error above is what Maven reports.

For JRuby's own build the range doesn't change anything. Maven always picks the top of the range, and that is exactly what jopenssl/version.rb asks require_jar for at runtime (1.86, and 1.86.1 for bctls).

Proposal: pin the versions in JRuby's build only. The ranges in the jruby-openssl gem stay as they are, so apps resolving jars through jar-dependencies can still pick any version their policies allow.

--- a/lib/pom.rb
+++ b/lib/pom.rb
@@ -196,6 +196,16 @@ project 'JRuby Lib Setup' do
     end
   end
 
+  # jruby-openssl declares Bouncy Castle as version ranges for its users; pin the
+  # build to the versions its jopenssl/version.rb loads so Maven does not have to
+  # fetch maven-metadata.xml for the ranges. Update together with jruby-openssl.
+  dependency_management do
+    jar 'org.bouncycastle:bcprov-jdk18on:1.86'
+    jar 'org.bouncycastle:bcpkix-jdk18on:1.86'
+    jar 'org.bouncycastle:bcutil-jdk18on:1.86'
+    jar 'org.bouncycastle:bctls-jdk18on:1.86.1'
+  end
+
   default_gemnames = default_gems.collect(&:first)
   all_gems = default_gems + bundled_gems

(plus the regenerated lib/pom.xml).

I checked it locally with ./mvnw -X -U -pl lib dependency:tree -Dincludes=org.bouncycastle. The resolved versions are the same (1.86:provided (version managed from [1.80,1.86])), and there are no Bouncy Castle maven-metadata.xml lookups. Without the pin the debug log has 88 of them.

The catch is that this has to be bumped together with jruby-openssl. If they get out of sync, the jars the build ships won't match the versions jopenssl/load.rb asks require_jar for. A small check could also compare the pins against JOpenSSL.version after the gem is installed.

@headius @kares does this sound OK, or would you rather handle it another way (e.g. in jruby-openssl's metadata, or just through CI retries)? I'm separately making tool/ci-retry.sh retry this error and pass -U on the retry, but that only works around the problem.

Activity

  1. kares commented on Nov 11, 2026

    @kares
    Member

    Sounds like a work-around, what happens if the pin stays in pom.rb while we update jruby-openssl where the version is now effectively out of sync? (hopefully just a build failure)

    Maybe there's a way of working with the tool (Maven), instead of trying to work-around it...

  2. headius commented on Nov 11, 2026

    @headius
    Member

    Yeah I have the same concern about this getting stale.

  3. zev commented on Nov 12, 2026

    @zev
    ContributorAuthor

    Fair point, both of you. I dropped the pin and looked at why Maven resolves the BC ranges at all.

    Bouncy Castle: use the jars the gem already ships

    The jruby-openssl gem already contains the exact jars jopenssl/version.rb loads, under vendor/org/bouncycastle/.... We only copy the gem's lib/ into stdlib and resolve BC separately through the ranges in its gemspec. If we exclude BC from the gem dependency and copy vendor/ too, the BC version comes from whatever jruby-openssl release we bundle. Bumping jruby-openssl bumps BC, and there's no pin to go stale.

    -  ['jruby-openssl', '0.19.5'],
    +  ['jruby-openssl', '0.19.5', { require_paths: ['lib', 'vendor'] }],
     ...
         dependency 'rubygems', name, version, type: 'gem', scope: :provided do
           exclusion 'rubygems:jar-dependencies'
    +      exclusion 'org.bouncycastle:*' if name == 'jruby-openssl'
         end

    With this, Maven makes no BC maven-metadata.xml requests, down from 12. The jars that land in stdlib are byte-identical to the ones from Central, and require 'openssl' loads BC 1.86 from stdlib. (Correction to the issue text: the "88 lookups" was a count of debug log lines. The real number of BC requests was 12.)

    On CI: the failures this issue is about were all BC, 11 jobs across 9 master runs since Sep 29. In 6 of them BC was the only error; 2 of those were Windows jobs, which tool/ci-retry.sh doesn't wrap. The other 5 also hit Central 429/403s and failed on the jruby-readline pom, so they still need retry/backoff.

    The gems have the same problem

    While measuring this, I noticed the BC ranges are only a small part of it. mavengem turns gem-to-gem dependencies into ranges too (e.g. net-http → uri [0.11.1,)), and those make up 108 of the 134 metadata requests in the lib build. Every gem the build resolves is already listed explicitly in lib/pom.rb, so these ranges never change what gets resolved; they only cost lookups. We already exclude rubygems:jar-dependencies this way, and excluding rubygems:* covers the rest:

         dependency 'rubygems', name, version, type: 'gem', scope: :provided do
    -      exclusion 'rubygems:jar-dependencies'
    +      exclusion 'rubygems:*'

    (in both the default-gem and bundled-gem loops)

    Measured with ./mvnw -X -U -pl lib dependency:tree:

    maven-metadata.xml requests
    master 134 (108 gems, 12 BC)
    with both exclusions 14 (none for gems or BC)

    That doesn't fix the CI failures above, which were all BC, but cutting ~120 requests per build may ease the rate limiting. I haven't measured that.

    There is one trade-off, and it's the same kind of concern you raised. If a gem bump adds a new gem dependency, Maven won't pull it in anymore; someone has to add it to lib/pom.rb. (Today Maven quietly adds it at the top of its range, which is the same range problem.) So that it can't go stale silently, I added a check at the end of the install_gems step. It fails the build if any shipped gem has a runtime dependency that the installed gems don't satisfy. It passes on master's gem set. With uri removed, it fails with:

    gem dependencies missing from lib/pom.rb (add them to default_gems or bundled_gems):
      net-http-0.10.0 depends on uri (>= 0.11.1)
      open-uri-0.5.0 depends on uri (>= 0)
    

    With both changes, a clean ./mvnw -pl lib clean package produces the same lib/ruby, bin/, and stdlib jar as master, byte for byte. The only difference is that 8 Maven bookkeeping files next to the BC jars (_remote.repositories, maven-metadata-local.xml) are no longer copied in.

    Does this look like the right direction? I can open one PR, or split BC and the gem exclusion into two if that's easier to review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions