SuccessChanges

Summary

  1. [SPARK-32556][INFRA][2.4] Fix release script to URL encode the user (details)
Commit a69396079db77ae0383b83963674f8213e2b0460 by prashant
[SPARK-32556][INFRA][2.4] Fix release script to URL encode the user
provided passwords
### What changes were proposed in this pull request? 1. URL encode the
`ASF_PASSWORD` of the release manager. 2. force delete a `.gitignore`
file as it may be absent, specially in the subsequent runs of the
script. And causes the release script failure. 3. Update the image to
install qpdf and jq dependency. 4. Increase the JVM HEAP memory for
MAVEN build.
### Why are the changes needed? Release script takes hours to run, and
if a single failure happens about somewhere midway, then either one has
to get down to manually doing stuff or re run the entire script. (This
is my understanding) So, I have made the fixes of a few failures,
discovered so far.
1. If the release manager password contains a char, that is not allowed
in URL, then it fails the build at the clone spark step.
`git clone "https://$ASF_USERNAME:$ASF_PASSWORD$ASF_SPARK_REPO" -b
$GIT_BRANCH`
          ^^^ Fails with bad URL
`ASF_USERNAME` may not be URL encoded, but we need to encode
`ASF_PASSWORD`.
2. If the `.gitignore` file is missing, it fails the build at `rm
.gitignore` step.
3. Ran into out of memory issues.
```
[INFO] Spark Integration for Kafka 0.10 ................... SUCCESS [
57.801 s]
[INFO] Kafka 0.10+ Source for Structured Streaming ........ SUCCESS
[01:13 min]
[INFO] Spark Kinesis Integration .......................... SUCCESS [
59.862 s]
[INFO] Spark Project Examples ............................. SUCCESS
[02:10 min]
[INFO] Spark Integration for Kafka 0.10 Assembly .......... SUCCESS [
45.454 s]
[INFO] Spark Avro ......................................... SUCCESS
[01:58 min]
[INFO] Spark Project External Flume Sink .................. SUCCESS
[01:34 min]
[INFO] Spark Project External Flume ....................... FAILURE
[20:04 min]
[INFO] Spark Project External Flume Assembly .............. SKIPPED
[INFO] Spark Project Kinesis Assembly 2.4.7 ............... SKIPPED
[INFO]
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 54:10 min
[INFO] Finished at: 2020-08-06T10:10:12Z
[INFO]
------------------------------------------------------------------------
[ERROR] Java heap space -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the
-e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError
```
### Does this PR introduce _any_ user-facing change? No
### How was this patch tested?
By running the release for branch-2.4, using both type of passwords,
i.e. passwords with special chars and without it.
For other branches, will followup in other PRs targeted for those
branches.
Closes #29371 from ScrapCodes/release-script-fixs.
Lead-authored-by: Prashant Sharma <prashsh1@in.ibm.com> Co-authored-by:
Prashant Sharma <prashant@apache.org> Signed-off-by: Prashant Sharma
<prashant@apache.org>
The file was modifieddev/create-release/release-build.sh (diff)
The file was modifieddev/create-release/release-tag.sh (diff)
The file was modifieddev/create-release/release-util.sh (diff)
The file was modifieddev/create-release/spark-rm/Dockerfile (diff)