Skip to content

Commit 9ddae41

Browse files
Merge pull request #167 from apache/deployment-changes
Add custom deploy script
2 parents 6494a09 + 3969a1f commit 9ddae41

17 files changed

+121
-75
lines changed

.github/workflows/maven.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -106,7 +106,7 @@ jobs:
106106
# The GitTag for CI purposes is irrelevant
107107
- name: Custom build script
108108
run: |
109-
./tools/scripts/compile-package-jar.sh $JAVA_HOME x.y.z .
109+
./tools/scripts/package-single-release-jar.sh $JAVA_HOME x.y.z .
110110
shell: bash
111111

112112
- name: Test & Report

.gitignore

+1-1
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ build.xml
6060
*.properties
6161
*.releaseBackup
6262
*.next
63-
*.tag
63+
*.asc
6464

6565
# Jekyll
6666
_site/

README.md

+8-5
Original file line numberDiff line numberDiff line change
@@ -151,14 +151,14 @@ This will create the following Jars:
151151

152152
#### Building for a specific java version
153153

154-
A build script named **compile-package-jar.sh** has been provided to package a JAR for a specific java version. This is necessary in cases where a developer is unable to install all the required versions of the JDK that are required as part of the Maven build.
154+
A build script named **package-single-release-jar.sh** has been provided to package a JAR for a specific java version. This is necessary in cases where a developer is unable to install all the required versions of the JDK that are required as part of the Maven build.
155155

156156
The build script performs the following steps:
157157

158158
1. Sets up staging directories under **target/** for the package files
159159
2. Uses git commands to gather information about the current Git commit and branch
160160
3. Compiles java source tree
161-
4. Packages a JAR containing compiled sources together with the Manifest, Licence and Notice files
161+
4. Packages a JAR containing compiled sources together with the Manifest, License and Notice files
162162
5. Checks and tests the assembled JAR by using the API to access four different resource types
163163

164164
The build script is located in the **tools/scripts/** directory and requires the following arguments:
@@ -171,15 +171,15 @@ For example, if the project base directory is `/src/datasketches-memory`;
171171

172172
To run the script for a release version:
173173

174-
./tools/scripts/compile-package-jar.sh $JAVA_HOME 2.1.0 /src/datasketches-memory
174+
./tools/scripts/package-single-release-jar.sh $JAVA_HOME 2.1.0 /src/datasketches-memory
175175

176176
To run the script for a snapshot version:
177177

178-
./tools/scripts/compile-package-jar.sh $JAVA_HOME 2.2.0-SNAPSHOT /src/datasketches-memory
178+
./tools/scripts/package-single-release-jar.sh $JAVA_HOME 2.2.0-SNAPSHOT /src/datasketches-memory
179179

180180
To run the script for an RC version:
181181

182-
./tools/scripts/compile-package-jar.sh $JAVA_HOME 2.1.0-RC1 /src/datasketches-memory
182+
./tools/scripts/package-single-release-jar.sh $JAVA_HOME 2.1.0-RC1 /src/datasketches-memory
183183

184184
Note that the script does **not** use the _Git Version Tag_ to adjust the working copy to a remote tag - it is expected that the user has a pristine copy of the desired branch/tag available **before** using the script.
185185

@@ -200,3 +200,6 @@ In order to build and contribute to this project, please read the relevant IDE d
200200

201201
- [Eclipse IDE Setup](docs/eclipse.md)
202202
- [IntelliJ IDE Setup](docs/intellij.md)
203+
204+
For releasing to AppNexus, please use the `sign-deploy-jar.sh` script in the scripts directory.
205+
See the documentation within the script for usage instructions.

datasketches-memory-java11/pom.xml

+2-14
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@
2424

2525
<parent>
2626
<groupId>org.apache.datasketches</groupId>
27-
<artifactId>datasketches-memory-root</artifactId>
27+
<artifactId>datasketches-memory</artifactId>
2828
<version>2.2.0</version>
2929
</parent>
3030

@@ -66,8 +66,7 @@
6666
</property>
6767
</activation>
6868
<properties>
69-
<assembly.script.path>${maven.multiModuleProjectDirectory}/tools/scripts/package-mr-jar.sh</assembly.script.path>
70-
<assembly.checks.script.path>${maven.multiModuleProjectDirectory}/tools/scripts/test-mr-jar.sh</assembly.checks.script.path>
69+
<assembly.script.path>${maven.multiModuleProjectDirectory}/tools/scripts/package-multi-release-jar.sh</assembly.script.path>
7170
<assembly.java.home>${env.JAVA_HOME}</assembly.java.home>
7271
<assembly.git.tag>${project.version}</assembly.git.tag>
7372
<assembly.base.dir>${maven.multiModuleProjectDirectory}</assembly.base.dir>
@@ -92,17 +91,6 @@
9291
<commandlineArgs>${assembly.script.path} ${assembly.java.home} ${assembly.git.tag} ${assembly.base.dir}</commandlineArgs>
9392
</configuration>
9493
</execution>
95-
<execution>
96-
<id>check-mr-jar</id>
97-
<goals>
98-
<goal>exec</goal>
99-
</goals>
100-
<phase>package</phase>
101-
<configuration>
102-
<executable>bash</executable>
103-
<commandlineArgs>${assembly.checks.script.path} ${assembly.java.home} ${assembly.git.tag} ${assembly.base.dir}</commandlineArgs>
104-
</configuration>
105-
</execution>
10694
</executions>
10795
</plugin>
10896
</plugins>

datasketches-memory-java8/pom.xml

+1-1
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@
2424

2525
<parent>
2626
<groupId>org.apache.datasketches</groupId>
27-
<artifactId>datasketches-memory-root</artifactId>
27+
<artifactId>datasketches-memory</artifactId>
2828
<version>2.2.0</version>
2929
</parent>
3030

docs/eclipse.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ for a Maven module should be set as follows:
3434

3535
| Maven submodule | JDK |
3636
| -------------------------- | --- |
37-
| datasketches-memory-root | 8 |
37+
| datasketches-memory | 8 |
3838
| datasketches-memory-java8 | 8 |
3939
| datasketches-memory-java11 | 11 |
4040

docs/intellij.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ for a maven module should be set as follows:
3535

3636
| Maven submodule | JDK |
3737
| -------------------------- | --- |
38-
| datasketches-memory-root | 8 |
38+
| datasketches-memory | 8 |
3939
| datasketches-memory-java8 | 8 |
4040
| datasketches-memory-java11 | 11 |
4141

docs/maven.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@
2020
# Maven Configuration: Multi-Module Project
2121

2222
This project is a multi-module Maven project. A multi-module Maven project consists of an
23-
aggregator project (the `datasketches-memory-root` project), together with a set of submodules.
23+
aggregator project (the `datasketches-memory` project), together with a set of submodules.
2424
The aggregator's configuration is inherited by each submodule, thus reducing duplication.
2525

2626
Datasketches Memory makes use of some features of the Java platform, for example, `Unsafe`,

docs/module-system.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ dependency graph.
4949

5050
#### org.apache.datasketches.memory
5151

52-
In the `datasketches-memory-java11` maven submodule root, the following module declaration has
52+
In the `datasketches-memory-java11` maven submodule source root, the following module declaration has
5353
been added:
5454

5555
```java

pom.xml

+2-2
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ under the License.
3131
</parent>
3232

3333
<groupId>org.apache.datasketches</groupId>
34-
<artifactId>datasketches-memory-root</artifactId>
34+
<artifactId>datasketches-memory</artifactId>
3535
<version>2.2.0</version>
3636
<!-- Required for multi-module project -->
3737
<packaging>pom</packaging>
@@ -325,7 +325,7 @@ under the License.
325325
<exclude>**/test-output/**/*</exclude>
326326
<exclude>**/img/**/*.png</exclude>
327327
<exclude>**/git.properties</exclude>
328-
<exclude>**/scripts/LoremIpsum.txt</exclude>
328+
<exclude>**/scripts/assets/LoremIpsum.txt</exclude>
329329
<exclude>LICENSE</exclude>
330330
<exclude>NOTICE</exclude>
331331
</excludes>
File renamed without changes.
File renamed without changes.
File renamed without changes.

tools/scripts/package-mr-jar.sh tools/scripts/package-multi-release-jar.sh

+4-2
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@
2020
# This is a general bash script to build a datasketches-memory-X.jar
2121
# with multi-release functionality. The sources, test-sources, tests and
2222
# javadoc jars are also included in the output.
23-
# It does use git and also uses the script getGitProperties.sh.
23+
# It does use git and also uses the script get-git-properties.sh.
2424
#
2525
# NOTE: This script assumes that `mvn package` has been run prior to invocation.
2626
# By default, it is called from the maven exec-plugin.
@@ -129,7 +129,7 @@ Created-By: Apache Datasketches Memory package-mr-jar.sh
129129
Multi-Release: true
130130
EOF
131131
#### Generate git.properties file ####
132-
echo "$($ScriptsDir/getGitProperties.sh $ProjectBaseDir $ProjectArtifactId $GitTag)" >> ${ArchiveDir}/MANIFEST.MF
132+
echo "$($ScriptsDir/get-git-properties.sh $ProjectBaseDir $ProjectArtifactId $GitTag)" >> ${ArchiveDir}/MANIFEST.MF
133133

134134
###########################
135135
#### MULTI-RELEASE JAR ####
@@ -185,3 +185,5 @@ prepare_jar $PackageJavaDoc
185185
rsync -q -a -I --filter="- .*" ${MemoryJava8Docs} $PackageJavaDoc
186186
${Jar_} cfm $OutputJavaDoc ${ArchiveDir}/MANIFEST.MF -C $PackageJavaDoc .
187187
echo "Created javadoc jar ${OutputJavaDoc}"
188+
189+
echo "$($ScriptsDir/test-jar.sh $JDKHome $GitTag $OutputMrJar $ProjectBaseDir)"

tools/scripts/compile-package-jar.sh tools/scripts/package-single-release-jar.sh

+7-24
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,8 @@
2323
# who do not wish to install several versions of the JDK on their
2424
# machine.
2525
# The script does not assume a POM file and does not use Maven.
26-
# It does use git and also uses the script getGitProperties.sh.
26+
# It does use git and also uses the script get-git-properties.sh and
27+
# test-jar.sh scripts.
2728

2829
# Required Input Parameters:
2930
# \$1 = absolute path of JDK home directory
@@ -46,7 +47,7 @@ ProjectBaseDir=$3 #this must be an absolute path
4647
#### Setup absolute directory references ####
4748
ProjectArtifactId="memory"
4849
ScriptsDir=${ProjectBaseDir}/tools/scripts/
49-
MemoryMapFile=$ScriptsDir/LoremIpsum.txt
50+
MemoryMapFile=$ScriptsDir/assets/LoremIpsum.txt
5051

5152
#### Initialise path dependent variables ####
5253
OutputDir=target
@@ -100,12 +101,12 @@ cp NOTICE $PackageMeta
100101
#### Generate MANIFEST.MF ####
101102
cat >> ${PackageManifest}<< EOF
102103
Manifest-Version: 1.0
103-
Created-By: Apache Datasketches Memory compile-package-jar.sh
104+
Created-By: Apache Datasketches Memory package-single-release-jar.sh
104105
Multi-Release: false
105106
EOF
106107

107108
#### Generate git.properties file ####
108-
echo "$($ScriptsDir/getGitProperties.sh $ProjectBaseDir $ProjectArtifactId $GitTag)" >> $PackageManifest
109+
echo "$($ScriptsDir/get-git-properties.sh $ProjectBaseDir $ProjectArtifactId $GitTag)" >> $PackageManifest
109110

110111
#### Copy source tree to target/src
111112
rsync -a -I $MemoryJava8Src $PackageSrc
@@ -134,30 +135,12 @@ echo
134135
echo "Building JAR from ${PackageContents}..."
135136
${Jar_} cfm $OutputJar ${PackageManifest} -C $PackageContents .
136137
echo
138+
echo "Successfully built ${OutputJar}"
137139

138140
# Uncomment this section to display JAR contents
139141
# echo "--- JAR CONTENTS ---"
140142
# echo
141143
# ${Jar_} tf ${OutputJar}
142144
# echo
143145

144-
echo "--- RUN JAR CHECKS ---"
145-
echo
146-
if [[ $JavaVersion -eq 8 ]]; then
147-
${Javac_} -cp $OutputJar -d $PackageTests $(find $ScriptsDir -name '*.java')
148-
${Java_} -cp $PackageTests:$OutputJar org.apache.datasketches.memory.tools.scripts.CheckMemoryJar $MemoryMapFile
149-
else
150-
${Javac_} \
151-
--add-modules org.apache.datasketches.memory \
152-
-p "$OutputJar" -d $PackageTests $(find $ScriptsDir -name '*.java')
153-
154-
${Java_} \
155-
--add-modules org.apache.datasketches.memory \
156-
--add-exports java.base/jdk.internal.misc=org.apache.datasketches.memory \
157-
--add-exports java.base/jdk.internal.ref=org.apache.datasketches.memory \
158-
--add-opens java.base/java.nio=org.apache.datasketches.memory \
159-
--add-opens java.base/sun.nio.ch=org.apache.datasketches.memory \
160-
-p $OutputJar -cp $PackageTests org.apache.datasketches.memory.tools.scripts.CheckMemoryJar $MemoryMapFile
161-
fi
162-
echo
163-
echo "Successfully built ${OutputJar}"
146+
echo "$($ScriptsDir/test-jar.sh $JDKHome $GitTag $OutputJar $ProjectBaseDir)"

tools/scripts/sign-deploy-jar.sh

+77
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
#!/bin/bash -e
2+
3+
# Licensed to the Apache Software Foundation (ASF) under one
4+
# or more contributor license agreements. See the NOTICE file
5+
# distributed with this work for additional information
6+
# regarding copyright ownership. The ASF licenses this file
7+
# to you under the Apache License, Version 2.0 (the
8+
# "License"); you may not use this file except in compliance
9+
# with the License. You may obtain a copy of the License at
10+
#
11+
# http://www.apache.org/licenses/LICENSE-2.0
12+
#
13+
# Unless required by applicable law or agreed to in writing,
14+
# software distributed under the License is distributed on an
15+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
16+
# KIND, either express or implied. See the License for the
17+
# specific language governing permissions and limitations
18+
# under the License.
19+
20+
# This is a general bash script to sign and deploy a datasketches-memory-X.jar.
21+
# This is intended to be used for releasing the Memory component to Maven central.
22+
23+
# Required Input Parameters:
24+
# \$1 = Git Version Tag for this deployment
25+
# Example tag for SNAPSHOT : 1.0.0-SNAPSHOT
26+
# Example tag for Release Candidate: 1.0.0-RC1
27+
# Example tag for Release : 1.0.0
28+
# \$2 = absolute path of project.basedir
29+
# For example: $ <this script>.sh 2.1.0 .
30+
31+
#### Extract GitTag, TestJar and ProjectBaseDir from input parameters ####
32+
GitTag=$1
33+
ProjectBaseDir=$2
34+
35+
#### Setup absolute directory references ####
36+
OutputDir=${ProjectBaseDir}/target
37+
38+
OutputMrJar=${OutputDir}/datasketches-memory-${GitTag}.jar
39+
OutputTests=${OutputDir}/datasketches-memory-${GitTag}-tests.jar
40+
OutputJavaDoc=${OutputDir}/datasketches-memory-${GitTag}-javadoc.jar
41+
OutputSources=${OutputDir}/datasketches-memory-${GitTag}-sources.jar
42+
OutputTestSources=${OutputDir}/datasketches-memory-${GitTag}-test-sources.jar
43+
OutputPom=${OutputDir}/datasketches-memory-${GitTag}-pom
44+
45+
#### Use GNU-GPG to create signature
46+
sign_file () {
47+
File=$1
48+
gpg --verbose --personal-digest-preferences=SHA512 --detach-sign -a $File
49+
}
50+
51+
### Deploy to nexus
52+
if [[ $GitTag == *SNAPSHOT ]]
53+
then
54+
echo "Using SNAPSHOT repository."
55+
DistributionsUrl=https://repository.apache.org/content/repositories/snapshots/
56+
DistributionsId=apache.snapshots.https
57+
else
58+
echo "Using RELEASES repository."
59+
DistributionsUrl=https://repository.apache.org/service/local/staging/deploy/maven2/
60+
DistributionsId=apache.releases.https
61+
fi;
62+
63+
mvn org.apache.maven.plugins:maven-gpg-plugin:3.0.1:sign-and-deploy-file \
64+
-Durl=$DistributionsUrl\
65+
-DrepositoryId=$DistributionsId \
66+
-Dfile=$OutputMrJar \
67+
-Dsources=$OutputSources \
68+
-Dfiles=$OutputTests,$OutputTestSources \
69+
-Dtypes=jar,jar \
70+
-Dclassifiers=tests,test-sources \
71+
-Djavadoc=$OutputJavaDoc \
72+
-Dpackaging=jar \
73+
-Dversion=$GitTag \
74+
-DupdateReleaseInfo=true \
75+
-DpomFile=${ProjectBaseDir}/pom.xml
76+
77+
echo "Successfully signed and deployed jars"

0 commit comments

Comments
 (0)