SuccessChanges

Summary

  1. [SPARK-30155][SQL] Rename parse() to parseString() to avoid conflict in (commit: a30ec19a7358f18849944ecfab1d2b14e733614c) (details)
  2. [SPARK-30148][SQL] Optimize writing plans if there is an analysis (commit: 51aa7a920ec097ed2a797687de8382e21691f18c) (details)
  3. [SPARK-30157][BUILD][TEST-HADOOP3.2][TEST-JAVA11] Upgrade Apache (commit: 1e0037b5e9ff077bdb59ad4536b7e5081a963089) (details)
  4. [SPARK-30156][BUILD] Upgrade Jersey from 2.29 to 2.29.1 (commit: afc4fa02bd2b7eb835e5c5dcbe0cbd1303910b42) (details)
  5. [SPARK-30147][SQL] Trim the string when cast string type to booleans (commit: e88d74052bf40eabab9e3388fa09e52097ffa3aa) (details)
  6. [SPARK-30163][INFRA] Use Google Maven mirror in GitHub Action (commit: 1068b8b24910eec8122bf7fa4748a101becf0d2b) (details)
  7. [SPARK-30163][INFRA][FOLLOWUP] Make `.m2` directory for cold start (commit: 16f1b23d75c0b44aac61111bfb2ae9bb0f3fab68) (details)
Commit a30ec19a7358f18849944ecfab1d2b14e733614c by dhyun
[SPARK-30155][SQL] Rename parse() to parseString() to avoid conflict in
Scala 2.13
### What changes were proposed in this pull request?
Rename internal method LegacyTypeStringParser.parse() to parseString().
### Why are the changes needed?
In Scala 2.13, the parse() definition clashes with supertype
declarations.
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
Existing tests.
Closes #26784 from srowen/SPARK-30155.
Authored-by: Sean Owen <sean.owen@databricks.com> Signed-off-by:
Dongjoon Hyun <dhyun@apple.com>
(commit: a30ec19a7358f18849944ecfab1d2b14e733614c)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/LegacyTypeStringParser.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala (diff)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala (diff)
Commit 51aa7a920ec097ed2a797687de8382e21691f18c by gurwls223
[SPARK-30148][SQL] Optimize writing plans if there is an analysis
exception
### What changes were proposed in this pull request? Optimized
QueryExecution.scala#writePlans().
### Why are the changes needed? If any query fails in Analysis phase and
gets AnalysisException, there is no need to execute further phases since
those will return a same result i.e, AnalysisException.
### Does this PR introduce any user-facing change? No
### How was this patch tested? Manually
Closes #26778 from amanomer/optExplain.
Authored-by: Aman Omer <amanomer1996@gmail.com> Signed-off-by:
HyukjinKwon <gurwls223@apache.org>
(commit: 51aa7a920ec097ed2a797687de8382e21691f18c)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala (diff)
Commit 1e0037b5e9ff077bdb59ad4536b7e5081a963089 by gurwls223
[SPARK-30157][BUILD][TEST-HADOOP3.2][TEST-JAVA11] Upgrade Apache
HttpCore from 4.4.10 to 4.4.12
### What changes were proposed in this pull request?
This PR aims to upgrade `Apache HttpCore` from 4.4.10 to 4.4.12.
### Why are the changes needed?
`Apache HttpCore v4.4.11` is the first official release for JDK11.
> This is a maintenance release that corrects a number of defects in
non-blocking SSL session code that caused compatibility issues with
TLSv1.3 protocol implementation shipped with Java 11.
For the full release note, please see the following.
-
https://www.apache.org/dist/httpcomponents/httpcore/RELEASE_NOTES-4.4.x.txt
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Pass the Jenkins.
Closes #26786 from dongjoon-hyun/SPARK-30157.
Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: HyukjinKwon
<gurwls223@apache.org>
(commit: 1e0037b5e9ff077bdb59ad4536b7e5081a963089)
The file was modifieddev/deps/spark-deps-hadoop-3.2-hive-2.3 (diff)
The file was modifiedpom.xml (diff)
The file was modifieddev/deps/spark-deps-hadoop-2.7-hive-2.3 (diff)
The file was modifieddev/deps/spark-deps-hadoop-2.7-hive-1.2 (diff)
Commit afc4fa02bd2b7eb835e5c5dcbe0cbd1303910b42 by dhyun
[SPARK-30156][BUILD] Upgrade Jersey from 2.29 to 2.29.1
### What changes were proposed in this pull request?
This PR aims to upgrade `Jersey` from 2.29 to 2.29.1.
### Why are the changes needed?
This will bring several bug fixes and important dependency upgrades.
-
https://eclipse-ee4j.github.io/jersey.github.io/release-notes/2.29.1.html
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Pass the Jenkins.
Closes #26785 from dongjoon-hyun/SPARK-30156.
Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: Dongjoon
Hyun <dhyun@apple.com>
(commit: afc4fa02bd2b7eb835e5c5dcbe0cbd1303910b42)
The file was modifieddev/deps/spark-deps-hadoop-2.7-hive-1.2 (diff)
The file was modifieddev/deps/spark-deps-hadoop-2.7-hive-2.3 (diff)
The file was modifiedpom.xml (diff)
The file was modifieddev/deps/spark-deps-hadoop-3.2-hive-2.3 (diff)
Commit e88d74052bf40eabab9e3388fa09e52097ffa3aa by yamamuro
[SPARK-30147][SQL] Trim the string when cast string type to booleans
### What changes were proposed in this pull request?
Now, we trim the string when casting string value to those `canCast`
types values, e.g. int, double, decimal, interval, date, timestamps,
except for boolean. This behavior makes type cast and coercion
inconsistency in Spark. Not fitting ANSI SQL standard either.
``` If TD is boolean, then Case: a) If SD is character string, then SV
is replaced by
   TRIM ( BOTH ' ' FROM VE )
   Case:
   i) If the rules for literal in Subclause 5.3, “literal”, can be
applied to SV to determine a valid value of the data type TD, then let
TV be that value.
  ii) Otherwise, an exception condition is raised: data exception —
invalid character value for cast. b) If SD is boolean, then TV is SV
``` In this pull request, we trim all the whitespaces from both ends of
the string before converting it to a bool value. This behavior is as
same as others, but a bit different from sql standard, which trim only
spaces.
### Why are the changes needed?
Type cast/coercion consistency
### Does this PR introduce any user-facing change?
yes, string with whitespaces in both ends will be trimmed before
converted to booleans.
e.g. `select cast('\t true' as boolean)` results `true` now, before this
pr it's `null`
### How was this patch tested?
add unit tests
Closes #26776 from yaooqinn/SPARK-30147.
Authored-by: Kent Yao <yaooqinn@hotmail.com> Signed-off-by: Takeshi
Yamamuro <yamamuro@apache.org>
(commit: e88d74052bf40eabab9e3388fa09e52097ffa3aa)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/StringUtils.scala (diff)
The file was modifiedsql/core/src/test/resources/sql-tests/results/cast.sql.out (diff)
The file was modifiedsql/core/src/test/resources/sql-tests/inputs/cast.sql (diff)
The file was modifieddocs/sql-migration-guide.md (diff)
The file was modifiedsql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastSuite.scala (diff)
Commit 1068b8b24910eec8122bf7fa4748a101becf0d2b by dhyun
[SPARK-30163][INFRA] Use Google Maven mirror in GitHub Action
### What changes were proposed in this pull request?
This PR aims to use [Google Maven
mirror](https://cloudplatform.googleblog.com/2015/11/faster-builds-for-Java-developers-with-Maven-Central-mirror.html)
in `GitHub Action` jobs to improve the stability.
```xml
<settings>
<mirrors>
   <mirror>
     <id>google-maven-central</id>
     <name>GCS Maven Central mirror</name>
   
<url>https://maven-central.storage-download.googleapis.com/repos/central/data/</url>
     <mirrorOf>central</mirrorOf>
   </mirror>
</mirrors>
</settings>
```
### Why are the changes needed?
Although we added Maven cache inside `GitHub Action`, the timeouts
happen too frequently during access `artifact descriptor`.
```
[ERROR] Failed to execute goal on project spark-mllib_2.12:
... Failed to read artifact descriptor for ...
... Connection timed out (Read failed) -> [Help 1]
```
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
This PR is irrelevant to Jenkins.
This is tested on the personal repository first. `GitHub Action` of this
PR should pass.
- https://github.com/dongjoon-hyun/spark/pull/11
Closes #26793 from dongjoon-hyun/SPARK-30163.
Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: Dongjoon
Hyun <dhyun@apple.com>
(commit: 1068b8b24910eec8122bf7fa4748a101becf0d2b)
The file was modified.github/workflows/master.yml (diff)
Commit 16f1b23d75c0b44aac61111bfb2ae9bb0f3fab68 by dhyun
[SPARK-30163][INFRA][FOLLOWUP] Make `.m2` directory for cold start
without cache
### What changes were proposed in this pull request?
This PR is a follow-up of https://github.com/apache/spark/pull/26793 and
aims to initialize `~/.m2` directory.
### Why are the changes needed?
In case of cache reset, `~/.m2` directory doesn't exist. It causes a
failure.
- `master` branch has a cache as of now. So, we missed this.
- `branch-2.4` has no cache as of now, and we hit this failure.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
This PR is tested against personal `branch-2.4`.
- https://github.com/dongjoon-hyun/spark/pull/12
Closes #26794 from dongjoon-hyun/SPARK-30163-2.
Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: Dongjoon
Hyun <dhyun@apple.com>
(commit: 16f1b23d75c0b44aac61111bfb2ae9bb0f3fab68)
The file was modified.github/workflows/master.yml (diff)