SuccessChanges

Summary

  1. [SPARK-25084][SQL][BACKPORT-2.3] distribute by" on multiple columns (commit: b9b35b9598dde5d8b2e47b40a0ca310527f31763) (details)
  2. [SPARK-25028][SQL] Avoid NPE when analyzing partition with NULL values (commit: 787790b3c733085b8b5e95cf832dedd481ab3b9a) (details)
Commit b9b35b9598dde5d8b2e47b40a0ca310527f31763 by wenchen
[SPARK-25084][SQL][BACKPORT-2.3] distribute by" on multiple columns
(wrap in brackets) may lead to codegen issue
## What changes were proposed in this pull request?
Backport #22066 to branch-2.3
Use different API in 2.3 here
```scala
|${ctx.JAVA_INT} $childResult = 0;
```
"distribute by" on multiple columns (wrap in brackets) may lead to
codegen issue.
Simple way to reproduce:
```scala
val df = spark.range(1000)
val columns = (0 until 400).map{ i => s"id as id$i" }
val distributeExprs = (0 until 100).map(c => s"id$c").mkString(",")
df.selectExpr(columns : _*).createTempView("test")
spark.sql(s"select * from test distribute by
($distributeExprs)").count()
```
## How was this patch tested?
UT in Jenkins
Closes #22077 from LantaoJin/SPARK-25084_2.3.
Authored-by: LantaoJin <jinlantao@gmail.com> Signed-off-by: Wenchen Fan
<wenchen@databricks.com>
(commit: b9b35b9598dde5d8b2e47b40a0ca310527f31763)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/hash.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala (diff)
Commit 787790b3c733085b8b5e95cf832dedd481ab3b9a by wenchen
[SPARK-25028][SQL] Avoid NPE when analyzing partition with NULL values
## What changes were proposed in this pull request?
`ANALYZE TABLE ... PARTITION(...) COMPUTE STATISTICS` can fail with a
NPE if a partition column contains a NULL value.
The PR avoids the NPE, replacing the `NULL` values with the default
partition placeholder.
## How was this patch tested?
added UT
Closes #22036 from mgaido91/SPARK-25028.
Authored-by: Marco Gaido <marcogaido91@gmail.com> Signed-off-by: Wenchen
Fan <wenchen@databricks.com>
(cherry picked from commit c220cc42abebbc98a6110b50f787eb6d338c2d97)
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
(commit: 787790b3c733085b8b5e95cf832dedd481ab3b9a)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzePartitionCommand.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/StatisticsCollectionSuite.scala (diff)