Skip to content

Enforce JDK 11 for development, but build Ingestion to target Java 8#518

Merged
feast-ci-bot merged 2 commits intofeast-dev:masterfrom
agoda-com:517-enforce-java-11-for-dev
Mar 28, 2020
Merged

Enforce JDK 11 for development, but build Ingestion to target Java 8#518
feast-ci-bot merged 2 commits intofeast-dev:masterfrom
agoda-com:517-enforce-java-11-for-dev

Conversation

@ches
Copy link
Copy Markdown
Member

@ches ches commented Mar 7, 2020

What this PR does / why we need it:

This is a PoC for the idea in #517 that Ingestion could be easily built for Java 8 while other components continue to target 11.

$ mvn clean compile

$ javap -v ingestion/target/classes/feast/ingestion/ImportJob.class | grep major
  major version: 52

$ javap -v serving/target/classes/feast/serving/ServingApplication.class | grep major
  major version: 55

Which issue(s) this PR fixes:

Addresses #517, speculatively and not completely.

Does this PR introduce a user-facing change?:

Feast Ingestion is again built for Java 8, for more certain Beam runner compatibility.

@ches
Copy link
Copy Markdown
Member Author

ches commented Mar 7, 2020

/assign @davidheryanto

@ches
Copy link
Copy Markdown
Member Author

ches commented Mar 7, 2020

Failure on String#strip() which is new in Java 11, guess that's a good sign 😇 Fixing…

@woop
Copy link
Copy Markdown
Member

woop commented Mar 9, 2020

Thanks for this @ches. I've commented on the issue. Happy to lgtm/approve if you are done.

@ches ches force-pushed the 517-enforce-java-11-for-dev branch from 2ee933b to 5ab68b7 Compare March 9, 2020 04:45
@ches
Copy link
Copy Markdown
Member Author

ches commented Mar 9, 2020

/test test-end-to-end-batch

@ches
Copy link
Copy Markdown
Member Author

ches commented Mar 9, 2020

Happy to lgtm/approve if you are done.

Functionally this is working for me, with light experimentation. Assuming agreement on #517, I don't feel there's anything more to do on this branch except maybe add a note similar to the last part of #517 (comment) to our development docs to clearly explain the policy and rationale.

It doesn't seem for me that IntelliJ automatically picked up the different <release> settings for maven-compiler-plugin to set per-module language levels automatically, I think that used to work for <source> and <target>. There may be some workflow or tool considerations like that to discover… interested to know if anyone finds any others.

@feast-ci-bot
Copy link
Copy Markdown
Collaborator

@ches: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
test-python-sdk 327835eba701460604a92774b0bc5672247b9d21 link /test test-python-sdk
test-end-to-end 327835eba701460604a92774b0bc5672247b9d21 link /test test-end-to-end

Full PR test history

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@woop
Copy link
Copy Markdown
Member

woop commented Mar 28, 2020

@ches would you mind rebasing this one? I don't think the tests will pass otherwise.

ches added 2 commits March 28, 2020 11:12
We build for Java 11 now, so the build will fail with older JDKs. Have
the Enforcer plugin do that for a more lucid error message.

This reverts 190e605 which was squash-merged in a larger commit.

Partially addresses feast-dev#517
As well as datatypes-java since ingestion depends on it.

Java 11 is desirable for the other components, but for Beam it may
impose limitations on what runners Feast can support, if it is even safe
to run on an 11 JRE now.

https://issues.apache.org/jira/browse/BEAM-2530

References feast-dev#517
@ches ches force-pushed the 517-enforce-java-11-for-dev branch from 327835e to c6d768c Compare March 28, 2020 05:38
@ches
Copy link
Copy Markdown
Member Author

ches commented Mar 28, 2020

Done, hadn't responded to the bot yet because I wanted to come back to adding the development guide info anyway. Also done now.


#### 5.2 IntelliJ Tips and Troubleshooting

For IntelliJ users, this section collects notes and setup recommendations for working comfortably on the Feast project, especially to coexist as peacefully as possible with the Maven build.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll move some other IntelliJ advice here that's duplicated (and out of sync) between the Contributing and Development guides, in another PR or GitBook edit.

I believe the thinking was that Contributing was getting unwieldy with technical setup / dev environment verification—I would agree, will take a look at moving things over with references left in Contributing. Probably this is also a legacy of duplicate CONTRIBUTING.md kept in the root awhile for GitHub.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yip, your assessment is correct. Lot's of little pebbles here, as you call them. We can chip away at it over time.

@woop
Copy link
Copy Markdown
Member

woop commented Mar 28, 2020

/approved
/lgtm

@feast-ci-bot
Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ches, woop

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@feast-ci-bot feast-ci-bot merged commit 880269b into feast-dev:master Mar 28, 2020
@ches ches deleted the 517-enforce-java-11-for-dev branch March 28, 2020 06:18
ches added a commit to agoda-com/feast that referenced this pull request May 21, 2020
With the addition of the Storage APIs and connector implementations, we
did not specify that their build should target Java 8 bytecode. Because
Ingestion depends on them, we regressed on keeping Ingestion compatible
with Java 8 runtimes according to feast-dev#518.

This could be confirmed prior to this change with, for example:

    $ mvn -pl ingestion clean compile
    $ javap -v storage/connectors/redis/target/classes/feast/storage/connectors/redis/writer/RedisCustomIO.class | grep major
      major version: 55

I take this as proof that a blacklist approach with 11 as the default in
our root POM is error prone, so this change flips it so that 8 is
default and our leaf applications that are able use 11 opt into it in
whitelist fashion.
woop pushed a commit that referenced this pull request May 22, 2020
…es (#722)

* Make storage modules target Java 8, since Ingestion must

With the addition of the Storage APIs and connector implementations, we
did not specify that their build should target Java 8 bytecode. Because
Ingestion depends on them, we regressed on keeping Ingestion compatible
with Java 8 runtimes according to #518.

This could be confirmed prior to this change with, for example:

    $ mvn -pl ingestion clean compile
    $ javap -v storage/connectors/redis/target/classes/feast/storage/connectors/redis/writer/RedisCustomIO.class | grep major
      major version: 55

I take this as proof that a blacklist approach with 11 as the default in
our root POM is error prone, so this change flips it so that 8 is
default and our leaf applications that are able use 11 opt into it in
whitelist fashion.

* Target Java 8 compatibility for Java SDK
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants