SuccessChanges

Summary

  1. [SPARK-29721][SQL] Prune unnecessary nested fields from Generate without (details)
  2. [SPARK-30639][BUILD] Upgrade Jersey to 2.30 (details)
  3. [SPARK-30579][DOC] Document ORDER BY Clause of SELECT statement in SQL (details)
Commit a0e63b61e7c5d55ae2a9213b95ab1e87ac7c203c by dhyun
[SPARK-29721][SQL] Prune unnecessary nested fields from Generate without
Project
### What changes were proposed in this pull request?
This patch proposes to prune unnecessary nested fields from Generate
which has no Project on top of it.
### Why are the changes needed?
In Optimizer, we can prune nested columns from Project(projectList,
Generate). However, unnecessary columns could still possibly be read in
Generate, if no Project on top of it. We should prune it too.
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
Unit test.
Closes #26978 from viirya/SPARK-29721.
Lead-authored-by: Liang-Chi Hsieh <liangchi@uber.com> Co-authored-by:
Liang-Chi Hsieh <viirya@gmail.com> Signed-off-by: Dongjoon Hyun
<dhyun@apple.com>
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/datasources/SchemaPruningSuite.scala (diff)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala (diff)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala (diff)
Commit 862959747ec3eb1f90d23ec91c1c6419468c9ea9 by dhyun
[SPARK-30639][BUILD] Upgrade Jersey to 2.30
### What changes were proposed in this pull request?
For better JDK11 support, this PR aims to upgrade **Jersey** and
**javassist** to `2.30` and `3.35.0-GA` respectively.
### Why are the changes needed?
**Jersey**: This will bring the following `Jersey` updates.
-
https://eclipse-ee4j.github.io/jersey.github.io/release-notes/2.30.html
- https://github.com/eclipse-ee4j/jersey/issues/4245 (Java 11
java.desktop module dependency)
**javassist**: This is a transitive dependency from 3.20.0-CR2 to
3.25.0-GA.
- `javassist` officially supports JDK11 from [3.24.0-GA release
note](https://github.com/jboss-javassist/javassist/blob/master/Readme.html#L308).
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Pass the Jenkins with both JDK8 and JDK11.
Closes #27357 from dongjoon-hyun/SPARK-30639.
Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: Dongjoon
Hyun <dhyun@apple.com>
The file was modifieddev/deps/spark-deps-hadoop-2.7-hive-2.3 (diff)
The file was modifieddev/deps/spark-deps-hadoop-3.2-hive-2.3 (diff)
The file was modifiedpom.xml (diff)
The file was modifieddev/deps/spark-deps-hadoop-2.7-hive-1.2 (diff)
Commit d5b92b24c41b047c64a4d89cc4061ebf534f0995 by yamamuro
[SPARK-30579][DOC] Document ORDER BY Clause of SELECT statement in SQL
Reference
### What changes were proposed in this pull request? Document ORDER BY
clause of SELECT statement in SQL Reference Guide.
### Why are the changes needed? Currently Spark lacks documentation on
the supported SQL constructs causing confusion among users who sometimes
have to look at the code to understand the usage. This is aimed at
addressing this issue.
### Does this PR introduce any user-facing change? Yes.
**Before:** There was no documentation for this.
**After.**
<img width="972" alt="Screen Shot 2020-01-19 at 11 50 57 PM"
src="https://user-images.githubusercontent.com/14225158/72708034-ac0bdf80-3b16-11ea-81f3-48d8087e4e98.png">
<img width="972" alt="Screen Shot 2020-01-19 at 11 51 14 PM"
src="https://user-images.githubusercontent.com/14225158/72708042-b0d09380-3b16-11ea-939e-905b8c031608.png">
<img width="972" alt="Screen Shot 2020-01-19 at 11 51 33 PM"
src="https://user-images.githubusercontent.com/14225158/72708050-b4fcb100-3b16-11ea-95d2-e4e302cace1b.png">
### How was this patch tested? Tested using jykyll build --serve
Closes #27288 from dilipbiswal/sql-ref-select-orderby.
Authored-by: Dilip Biswal <dkbiswal@gmail.com> Signed-off-by: Takeshi
Yamamuro <yamamuro@apache.org>
The file was modifieddocs/sql-ref-syntax-qry-select-orderby.md (diff)