Skip to content

Detect modules more robustly#6615

Merged
headius merged 1 commit intojruby:jruby-9.2from
headius:better_module_detect
Mar 15, 2021
Merged

Detect modules more robustly#6615
headius merged 1 commit intojruby:jruby-9.2from
headius:better_module_detect

Conversation

@headius
Copy link
Member

@headius headius commented Mar 12, 2021

Detecting module support based on the existence of the jmods
directory was fragile for at least two reasons:

Instead we detect modules using two other mechanisms that should
reliably work on all modularized Java runtimes:

  • Check for JAVA_HOME/lib/modules, generated by the jlink process.
  • Check for a JAVA_HOME/release file with a MODULES entry.

See https://stackoverflow.com/questions/66601929 for a summary of
detection mechanism.

Fixes #6608.

jruby-launcher will need a matching update.

@fzakaria
Copy link

fzakaria commented Mar 12, 2021

Nix's distribution does have these two mechanisms present.

❯ ls -l /nix/store/2xv65447vkq09bpx1mfci1cx2vdvp153-openjdk-headless-11.0.9+11/lib/openjdk/lib/modules
.r--r--r-- 141M fmzakari 31 Dec  1969 /nix/store/2xv65447vkq09bpx1mfci1cx2vdvp153-openjdk-headless-11.0.9+11/lib/openjdk/lib/modules
❯ ls -l /nix/store/2xv65447vkq09bpx1mfci1cx2vdvp153-openjdk-headless-11.0.9+11/lib/openjdk/release
.r--r--r-- 1.2k fmzakari 31 Dec  1969 /nix/store/2xv65447vkq09bpx1mfci1cx2vdvp153-openjdk-headless-11.0.9+11/lib/openjdk/release

The challenge was that the JAVA_HOME set in the wrapped script was wrong :(
Java itself seems to know it's own home directory, which I feel like should have been used.
(is there any point then of even having JAVA_HOME set?)

❯ JAVA_HOME=~/; java -XshowSettings:properties -version 2>&1 | grep java.home

    java.home = /nix/store/2xv65447vkq09bpx1mfci1cx2vdvp153-openjdk-headless-11.0.9+11/lib/openjdk

@fzakaria
Copy link

Here is a better example to demonstrate:

echo $JAVA_HOME
/home/fmzakari/
❯ java -XshowSettings:properties -version 2>&1 | grep java.home

    java.home = /nix/store/2xv65447vkq09bpx1mfci1cx2vdvp153-openjdk-headless-11.0.9+11/lib/openjdk

Seems like having the user not even have to set JAVA_HOME is even better!

@headius
Copy link
Member Author

headius commented Mar 12, 2021

The problem is that we need to pass module flags to the java command, so we need to determine whether modules are present before we start up. If we have to run java twice – once to determine if it supports modules and once to launch – we add quite a lot of time to the startup process:

$ time java -XshowSettings:properties -version
Property settings:
    awt.toolkit = sun.lwawt.macosx.LWCToolkit
    file.encoding = UTF-8
...
    user.name = headius
    user.timezone = 

openjdk version "11.0.4" 2019-07-16
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.4+11)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.4+11, mixed mode)

real	0m0.115s
user	0m0.116s
sys	0m0.029s

We really need to be able to tell whether the current Java version supports modules without running any commands (or if we run anything it has to be very fast).

We have various mechanisms to figure out the JAVA_HOME if none is set, but they depend on the java command resolving to something inside a typical Java distribution (i.e. in a bin dir where ../lib/modules is present).

I see your PR for Nix was accepted...shouldn't that combined with #6615 resolve this?

@fzakaria
Copy link

Oh totally, I mean it's resolved without this even since the Nix distribution does have the jmod directory.
I was just enjoying the discussion and giving feedback! :)

I forgot that a main goal is to reduce startup time. 👍

Detecting module support based on the existence of the jmods
directory was fragile for at least two reasons:

* The jmods directory is only present in some builds of the JDK
  (not JRE) that can be used to jlink new runtime environments.
* Some distributions of the JDK may omit or relocate this
  directory, such as Nix's version (jruby#6608).

Instead we detect modules using two other mechanisms that should
reliably work on all modularized Java runtimes:

* Check for JAVA_HOME/lib/modules, generated by the jlink process.
* Check for a JAVA_HOME/release file with a MODULES entry.

See https://stackoverflow.com/questions/66601929 for a summary of
detection mechanism.

Fixes jruby#6608.

jruby-launcher will need a matching update.
@headius headius force-pushed the better_module_detect branch from 3e715d9 to ebf92b3 Compare March 15, 2021 17:38
@headius
Copy link
Member Author

headius commented Mar 15, 2021

Modified the check to first confirm release file is there, to avoid grep error. I think this is ready to merge.

@headius headius merged commit 4e001a4 into jruby:jruby-9.2 Mar 15, 2021
@headius headius deleted the better_module_detect branch March 15, 2021 17:56
headius added a commit to jruby/jruby-launcher that referenced this pull request Mar 15, 2021
headius added a commit to jruby/jruby-launcher that referenced this pull request Mar 15, 2021
if [ -d "$JAVA_HOME"/jmods ]; then
is_java9=1
# Detect modularized Java
if [ -f ${JAVA_HOME}/lib/modules ] || [ -f ${JAVA_HOME}/release ] && grep -q ^MODULES ${JAVA_HOME}/release; then
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @headius, was the " drop around $JAVA_HOME intentional here? Asking, because I am seeing some errors in 9.2.17.0 which seems to be related to me having spaces in my $JAVA_HOME:

$ ruby --version
/Users/robin/.rubies/jruby-9.2.17.0/bin/ruby: line 144: [: /Library/Internet: binary operator expected
/Users/robin/.rubies/jruby-9.2.17.0/bin/ruby: line 144: [: /Library/Internet: binary operator expected
jruby 9.2.17.0 (2.5.8) 2021-03-29 84d363da97 Java HotSpot(TM) 64-Bit Server VM 25.281-b09 on 1.8.0_281-b09 +jit [darwin-x86_64]

$ unset JAVA_HOME

$ ruby --version 
jruby 9.2.17.0 (2.5.8) 2021-03-29 84d363da97 OpenJDK 64-Bit Server VM 15.0.1+9-18 on 15.0.1+9-18 +jit [darwin-x86_64]

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was not intentional and I even thought at one point I need to restore those quotes. You can make the modification yourself locally, or use the native launcher from jruby-launcher gem which should not have this issue. I will open a PR to fix this in JRuby proper.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW we do recommend not having the JDK installed in a path with spaces for many other reasons, including the complexity of making sure re-launched Java/JRuby subprocesses use the proper quoting. Thank you for bringing this issue to my attention again, I have pushed #6638 to fix it.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick turn-around.

FWIW we do recommend not having the JDK installed in a path with spaces for many other reasons

Hehe yes, that's why I thought the change might have been intentional. I might add that the path with spaces is "Oracle's JRE path on macOS", at least for Java 8.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@walro Could you share more information about your system in the PR? For me, JDKs are installed in /Library/Java/JavaVirtualMachines so this /Library/Internet Something path may be a new thing we have to be prepared for.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely, done so!

headius added a commit to headius/jruby that referenced this pull request Mar 30, 2021
headius added a commit to headius/jruby that referenced this pull request Mar 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants