Repository navigation
Update dependencies #342
Description
As mentioned in #169 (comment) and #157 (comment) it is preferrable if team members update dependencies:
The following dependencies have later milestone versions:
- com.github.jnr:jffi [1.3.9 -> 1.3.13]
http://github.com/jnr/jffi - com.github.jnr:jnr-constants [0.10.3 -> 0.10.4]
http://github.com/jnr/jnr-constants - com.github.jnr:jnr-ffi [2.2.12 -> 2.2.16]
http://github.com/jnr/jnr-ffi - com.github.jnr:jnr-posix [3.1.15 -> 3.1.19]
- com.google.guava:guava [32.1.2-jre -> 33.2.1-jre]
https://github.com/google/guava - com.ibm.icu:icu4j [71.1 -> 75.1]
https://icu.unicode.org/ - com.ibm.informix:jdbc [4.50.8 -> 4.50.11]
https://www.ibm.com/informix - com.oracle.database.jdbc:ojdbc8 [19.14.0.0 -> 23.4.0.24.05]
https://www.oracle.com/database/technologies/maven-central-guide.html - commons-io:commons-io [2.15.1 -> 2.16.1]
https://commons.apache.org/proper/commons-io/ - io.netty:netty-buffer [4.1.73.Final -> 4.1.111.Final]
http://netty.io/ - io.netty:netty-codec [4.1.73.Final -> 4.1.111.Final]
http://netty.io/ - io.netty:netty-common [4.1.73.Final -> 4.1.111.Final]
http://netty.io/ - io.netty:netty-handler [4.1.73.Final -> 4.1.111.Final]
http://netty.io/ - io.netty:netty-resolver [4.1.73.Final -> 4.1.111.Final]
http://netty.io/ - io.netty:netty-transport [4.1.73.Final -> 4.1.111.Final]
http://netty.io/ - jline:jline [2.14.5 -> 2.14.6]
- org.antlr:antlr-runtime [3.5.2 -> 3.5.3]
http://www.antlr.org - org.apache.ant:ant [1.10.12 -> 1.10.14]
https://ant.apache.org/ - org.apache.commons:commons-compress [1.26.0 -> 1.26.2]
https://commons.apache.org/proper/commons-compress/ - org.bouncycastle:bcpkix-jdk18on [1.77 -> 1.78.1]
https://www.bouncycastle.org/java.html - org.bouncycastle:bcprov-jdk18on [1.77 -> 1.78.1]
https://www.bouncycastle.org/java.html - org.ow2.asm:asm [9.6 -> 9.7]
http://asm.ow2.io/ - org.ow2.asm:asm-commons [9.6 -> 9.7]
http://asm.ow2.io/ - org.ow2.asm:asm-util [9.6 -> 9.7]
http://asm.ow2.io/
Activity
Stewori commentedon Jul 19, 2024
I am a bit hesistent to do these updates for 2.7.4 because beta phase is already somewhat through. There will be an rc phase though and these updates are not really changes to Jython. Given the long Jython release cycle, third party libs bundled with 2.7.4 should IMO be indeed as new as possible at release time, especially security-relevant ones like bouncycastle. @jeff5 if you consent I can tackle the third party libs update, but let's agree on a date to avoid repeated effort due to further new releases. My suggestion would be right before rc release (like one day before), to include all updates that pass regrtests. I assume it is okay to take the jars from maven central, because it is an annoying hassle to work through the individual homepages.
jeff5 commentedon Jul 19, 2024
Left to myself, I'd freeze dependencies at the beta. I did a round of updating then, but beta phase has run on a bit. I think accepting dependency changes into a release candidate, on condition they don't create a regression is a fair suggestion. If we have to make code changes of our own to adjust to them, that's a beta.
@thetric thanks for the comprehensive list. I know I tried to advance a few of of these at b1, and they caused problems. I can see I tried netty 4.1.80 and rejected it.
We don't bundle the Oracle and Informix JARs, but of course we specify versions.
Stewori commentedon Jul 19, 2024
Thanks, @jeff5 for the clarification. I was not aware that a netty update would cause problems (perhaps you pointed it out somewhere but I forgot or overlooked). Perhaps it would be good to directly open issues for such failures to keep track (may be a "won't fix", that would also be fine, but it should be tracked).
jeff5 commentedon Jul 19, 2024
I would have said I'd noted it in the build files, but not when I look. (Maybe on a branch I dropped.)
jeff5 commentedon Jul 20, 2024
I'll work through these and see how they treat us.
jeff5 commentedon Jul 21, 2024
JARs for jdbc and ojdbc are citations in our build, but not bundled.
Ant is used in the build (including for a custom plug-in) but not bundled.
Stewori commentedon Jul 21, 2024
Not bundling ant appears a bit dubious to me, because I get plenty of missing symbol and missing package errors if ant is not on the classpath, e.g. like this one:
This does not look like code that can never execute on runtime, does it?
See
org.apache.tools.antoccurring in the imports ofExposeTask
GlobMatchingTask
JythoncAntTask
JycompileAntTask
Maybe the
org.python.utilpackage needs some cleanup (for the runtime distribution). No need to handle this now before the release, but maybe we should track it in an issue.JythoncAnTaskis probably obsolete and I agree thatJycompileAntTasksounds like only build-time. ButExposeTask?jeff5 commentedon Jul 21, 2024
I believe it is what this is about:
jython/build.xml
Line 750 in 82bd117
ExposeTaskprocesses Java classes andJycompileAntTaskcompiles the stdlib.Ideally, build tools would be in a separate tree, not mixed in with the delivered code, and filtered out later.
jython/build.xml
Line 982 in 82bd117
Stewori commentedon Jul 21, 2024
After some reading on the topic I learned that it is quite more usual for Java programs/libs to omit dependencies as "compile-time only" than I thought. Seemingly, there is no systematic/general/reliable/automatic way to decide which libs are runtime dependencies and which are not (I know, the dependencies in pom files may indicate the dependency types, but I found it unreliable, at least for determining compile-time dependencies). So it's often a bit dubious, perhaps dependent on the use case, and some unfortunate user may still find a way to use a lib such that a missing runtime-lib/thought-to-be compile-time lib crashes the thing. (but of course you know all this)
Given that issue, I suppose some cleanup would be good. What can safely be sorted out should be, to minimize the issue at least.
Yes, that's also my thinking. Probably, the first step would be to move everything ant-dependent out of
org.python.utilinto a new folder and package, e.g.org.python.antwhich can then be added toexcludesinbuild.xml. Perhaps, we can attempt that refactoring for 2.7.5. Would be good to have the separation for Jython 3 at least. Can Java modularization be of any help for that? (Note: For Java modularization compile-time-only dependencies are referenced asrequires static.) Or maybe the problem is obsolete in Gradle.jeff5 commentedon Jul 22, 2024
My appetite for improvements to Jython 2 is limited. I looked into the Ant plug-ins for a few hours and decided to leave them alone.
I'm writing from knowledge of Jython 3 (or VSJ) and struggling with Gradle, which has a specific feature for tool-building before the build proper. Done right, it should cut down the build time by recognising when artefacts are up to date. I made it work once in Java, but I can't figure it out when the tool is in Python. I've spent more time trying than it will ever save.
No. It needs you to tell it, but how you do that makes reasonable sense.
jeff5 commentedon Jul 22, 2024
I tried updating Netty to 4.1.112, but it gives me errors in
test_httplibandtest_ssl_jy. The error is always:I smell concurrency/async, which is Netty's selling point, but I believe we do not need it for the library and have not used it correctly.
Stewori commentedon Jul 22, 2024
At least there won't be AntTasks.
jeff5 commentedon Jul 28, 2024
I think we have updated all we can in response to this, so it looks like we have an rc1. @Stewori : am I missing something you really wanted to be in?
Stewori commentedon Jul 29, 2024
Thank you Jeff, for all this work. I have nothing more to add here. The Netty situation is not ideal, but apparently not solvable in short term. So, feel free to proceed!
jeff5 commentedon Aug 2, 2024
As the Netty shortfall is documented in #349 I'll close this.