Skip to content

Java SDK release script #406

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
feast-ci-bot merged 11 commits into from
Jan 7, 2020
Merged

Java SDK release script #406

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
bb2f07d
Use back revision variable in pom.xml
davidheryanto Jan 4, 2020
faee6fa
Add flatten maven plugin
davidheryanto Jan 4, 2020
0eaecd0
Increase versions for maven source,javadoc,spotless plugins
davidheryanto Jan 4, 2020
f2294c9
Add gpg-plugin needed to sign releases
davidheryanto Jan 4, 2020
07ebfa8
Use oss configure for flatten plugin, add developers info in pom.xml …
davidheryanto Jan 4, 2020
cdc5065
Add publish-java-sdk script
davidheryanto Jan 4, 2020
0a956fe
Add more logs to publish-java-sdk.sh
davidheryanto Jan 4, 2020
6240ce3
Add ProwJob publish-java-sdk
davidheryanto Jan 4, 2020
b38166a
Use GPG_KEY_IMPORT_DIR variable
davidheryanto Jan 4, 2020
0731188
Update revision in pom.xml to 0.4.2-SNAPSHOT
davidheryanto Jan 6, 2020
0538e79
Merge branch 'master' into java-sdk-release-automation
davidheryanto Jan 6, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,4 +174,8 @@ dmypy.json
.pyre/
.vscode

sdk/python/docs/html
# .flattened-pom.xml is generated by flatten-maven-plugin.
# This pom should not be committed because it is only used during release / deployment.
.flattened-pom.xml

sdk/python/docs/html
29 changes: 29 additions & 0 deletions .prow/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,34 @@ postsubmits:
# https://github.com/semver/semver/issues/232
- ^v(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(-(0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(\.(0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*)?(\+[0-9a-zA-Z-]+(\.[0-9a-zA-Z-]+)*)?$

- name: publish-java-sdk
decorate: true
spec:
containers:
- image: maven:3.6-jdk-8
command:
- bash
- -c
- .prow/scripts/publish-java-sdk.sh --revision ${PULL_BASE_REF:1}
volumeMounts:
- name: gpg-keys
mountPath: /etc/gpg
readOnly: true
- name: maven-settings
mountPath: /root/.m2/settings.xml
subPath: settings.xml
readOnly: true
volumes:
- name: gpg-keys
secret:
secretName: gpg-keys
- name: maven-settings
secret:
secretName: maven-settings
branches:
# Filter on tags with semantic versioning, prefixed with "v"
- ^v(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(-(0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(\.(0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*)?(\+[0-9a-zA-Z-]+(\.[0-9a-zA-Z-]+)*)?$

- name: publish-docker-images
decorate: true
spec:
Expand Down Expand Up @@ -278,4 +306,5 @@ postsubmits:
secret:
secretName: feast-service-account
branches:
# Filter on tags with semantic versioning, prefixed with "v"
- ^v(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(-(0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(\.(0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*)?(\+[0-9a-zA-Z-]+(\.[0-9a-zA-Z-]+)*)?$
72 changes: 72 additions & 0 deletions .prow/scripts/publish-java-sdk.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
#!/usr/bin/env bash

set -e
set -o pipefail

GPG_KEY_IMPORT_DIR=/etc/gpg

usage()
{
echo "usage: publish-java-sdk.sh

--revision Value for the revision e.g. '0.2.3'
--gpg-key-import-dir Directory containing existing GPG keys to import.
The directory should contain these 2 files:
- public-key
- private-key
The default value is '/etc/gpg'

This script assumes the GPG private key is protected by a passphrase.
The passphrase can be specified in \$HOME/.m2/settings.xml. In the same xml
file, credentials to upload releases to Sonatype must also be provided.

# Example settings: ~/.m2/settings.xml
<settings>
<servers>
<server>
<id>ossrh</id>
<username>SONATYPE_USER</username>
<password>SONATYPE_PASSWORD</password>
</server>
</servers>
<profiles>
<profile>
<id>ossrh</id>
<properties>
<gpg.passphrase>GPG_PASSPHRASE</gpg.passphrase>
</properties>
</profile>
</profiles>
</settings>
"
}

while [ "$1" != "" ]; do
case "$1" in
--revision ) REVISION="$2"; shift;;
--gpg-key-import-dir ) GPG_KEY_IMPORT_DIR="$2"; shift;;
-h | --help ) usage; exit;;
* ) usage; exit 1
esac
shift
done

if [ -z $REVISION ]; then usage; exit 1; fi

echo "============================================================"
echo "Checking Maven and GPG versions"
echo "============================================================"
mvn --version
echo ""
gpg --version

echo "============================================================"
echo "Importing GPG keys"
echo "============================================================"
gpg --import --batch --yes $GPG_KEY_IMPORT_DIR/public-key
gpg --import --batch --yes $GPG_KEY_IMPORT_DIR/private-key

echo "============================================================"
echo "Deploying Java SDK with revision: $REVISION"
echo "============================================================"
mvn --projects sdk/java -Drevision=$REVISION --batch-mode clean deploy
2 changes: 1 addition & 1 deletion core/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
<parent>
<groupId>dev.feast</groupId>
<artifactId>feast-parent</artifactId>
<version>0.3.6-SNAPSHOT</version>
<version>${revision}</version>
</parent>

<name>Feast Core</name>
Expand Down
2 changes: 1 addition & 1 deletion ingestion/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
<parent>
<groupId>dev.feast</groupId>
<artifactId>feast-parent</artifactId>
<version>0.3.6-SNAPSHOT</version>
<version>${revision}</version>
</parent>

<name>Feast Ingestion</name>
Expand Down
109 changes: 92 additions & 17 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@

<groupId>dev.feast</groupId>
<artifactId>feast-parent</artifactId>
<version>0.3.6-SNAPSHOT</version>
<version>${revision}</version>
<packaging>pom</packaging>

<modules>
Expand All @@ -35,6 +35,7 @@
</modules>

<properties>
<revision>0.4.2-SNAPSHOT</revision>
<github.url>https://github.com/gojek/feast</github.url>

<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
Expand All @@ -47,7 +48,6 @@
<org.apache.beam.version>2.16.0</org.apache.beam.version>
<com.google.cloud.version>1.91.0</com.google.cloud.version>
<io.prometheus.version>0.8.0</io.prometheus.version>

<byte-buddy.version>1.9.10</byte-buddy.version>
<hamcrest.version>1.3</hamcrest.version>
<kafka.version>2.3.0</kafka.version>
Expand All @@ -58,9 +58,18 @@

<organization>
<name>Gojek</name>
<url>https://www.gojek.io/</url>
<url>https://www.gojek.com</url>
</organization>

<developers>
<developer>
<name>Feast Authors</name>
<url>${github.url}</url>
<organization>Gojek</organization>
<organizationUrl>https://www.gojek.com</organizationUrl>
</developer>
</developers>

<licenses>
<license>
<name>Apache License, Version 2.0</name>
Expand All @@ -81,11 +90,16 @@
<url>${github.url}/issues</url>
</issueManagement>

<!-- Release Java library on Sonatype https://central.sonatype.org/pages/apache-maven.html -->
<distributionManagement>
<snapshotRepository>
<id>ossrh</id>
<url>https://oss.sonatype.org/content/repositories/snapshots</url>
</snapshotRepository>
<repository>
<id>ossrh</id>
<url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
</repository>
</distributionManagement>

<dependencyManagement>
Expand Down Expand Up @@ -274,21 +288,10 @@
</extensions>

<plugins>
<plugin>
<groupId>org.sonatype.plugins</groupId>
<artifactId>nexus-staging-maven-plugin</artifactId>
<version>1.6.7</version>
<extensions>true</extensions>
<configuration>
<serverId>ossrh</serverId>
<nexusUrl>https://oss.sonatype.org/</nexusUrl>
<autoReleaseAfterClose>false</autoReleaseAfterClose>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<version>2.2.1</version>
<version>3.2.1</version>
<executions>
<execution>
<id>attach-sources</id>
Expand All @@ -301,7 +304,7 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.9.1</version>
<version>3.1.1</version>
<executions>
<execution>
<id>attach-javadocs</id>
Expand All @@ -314,7 +317,7 @@
<plugin>
<groupId>com.diffplug.spotless</groupId>
<artifactId>spotless-maven-plugin</artifactId>
<version>1.26.0</version>
<version>1.26.1</version>
<configuration>
<java>
<licenseHeader>
Expand Down Expand Up @@ -429,6 +432,78 @@
<skip>true</skip>
</configuration>
</plugin>
<!-- nexus-staging-maven-plugin configures Maven to deploy to OSSRH Nexus Repository Manager -->
<plugin>
<groupId>org.sonatype.plugins</groupId>
<artifactId>nexus-staging-maven-plugin</artifactId>
<version>1.6.8</version>
<extensions>true</extensions>
<configuration>
<serverId>ossrh</serverId>
<nexusUrl>https://oss.sonatype.org/</nexusUrl>
<!-- autoReleaseAfterClose is true as the release should be automated via continuous integration -->
<autoReleaseAfterClose>true</autoReleaseAfterClose>
</configuration>
</plugin>
<!--
flatten-maven-plugin is used to flatten the generated POM during deployment. This is
required when releasing a project with a parent dependency where the version
is using ${revision} variable. This plugin will resolve and flatten the relationship.
https://www.mojohaus.org/flatten-maven-plugin/index.html
-->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>flatten-maven-plugin</artifactId>
<version>1.1.0</version>
<configuration>
<flattenMode>oss</flattenMode>
</configuration>
<executions>
<execution>
<id>flatten</id>
<phase>process-resources</phase>
<goals>
<goal>flatten</goal>
</goals>
</execution>
<execution>
<id>flatten.clean</id>
<phase>clean</phase>
<goals>
<goal>clean</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- maven-gpg-plugin is used to sign Maven components. This is required when releasing libraries to Maven Central -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-gpg-plugin</artifactId>
<version>1.6</version>
<executions>
<execution>
<id>sign-artifacts</id>
<phase>verify</phase>
<goals>
<goal>sign</goal>
</goals>
<!--
This configuration helps with performing GPG operations non-interactively.
For example when releasing via continuous integration.
It is assumed that the GPG command used is version 2.x.
-->
<configuration>
<gpgArguments>
<arg>--pinentry-mode</arg>
<arg>loopback</arg>
</gpgArguments>
<!-- Setting to allow retrieval of GPG passphrase from ~/.m2/settings.xml, refer to https://maven.apache.org/plugins/maven-gpg-plugin/usage.html -->
<!--suppress MavenModelInspection -->
<passphrase>${gpg.passphrase}</passphrase>
</configuration>
</execution>
</executions>
</plugin>
</plugins>

<pluginManagement>
Expand Down
2 changes: 1 addition & 1 deletion sdk/java/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
<parent>
<groupId>dev.feast</groupId>
<artifactId>feast-parent</artifactId>
<version>0.3.6-SNAPSHOT</version>
<version>${revision}</version>
<relativePath>../..</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion serving/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
<parent>
<groupId>dev.feast</groupId>
<artifactId>feast-parent</artifactId>
<version>0.3.6-SNAPSHOT</version>
<version>${revision}</version>
</parent>

<artifactId>feast-serving</artifactId>
Expand Down