1. [SPARK-25371][SQL][BACKPORT-2.3] struct() should allow being called with (commit: d8ec5ff7502451f61f2be3871d9bd1461b26accd) (details)
  2. [SPARK-25402][SQL] Null handling in BooleanSimplification (commit: db9c041790ac3a0b883d0783cbdd12ab459758c2) (details)
Commit d8ec5ff7502451f61f2be3871d9bd1461b26accd by wenchen
[SPARK-25371][SQL][BACKPORT-2.3] struct() should allow being called with
0 args
## What changes were proposed in this pull request?
SPARK-21281 introduced a check for the inputs of `CreateStructLike` to
be non-empty. This means that `struct()`, which was previously
considered valid, now throws an Exception.  This behavior change was
introduced in 2.3.0. The change may break users' application on upgrade
and it causes `VectorAssembler` to fail when an empty `inputCols` is
The PR removes the added check making `struct()` valid again.
## How was this patch tested?
added UT
Closes #22391 from mgaido91/SPARK-25371_2.3.
Authored-by: Marco Gaido <> Signed-off-by: Wenchen
Fan <>
(commit: d8ec5ff7502451f61f2be3871d9bd1461b26accd)
The file was modifiedmllib/src/test/scala/org/apache/spark/ml/feature/VectorAssemblerSuite.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala (diff)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala (diff)
Commit db9c041790ac3a0b883d0783cbdd12ab459758c2 by wenchen
[SPARK-25402][SQL] Null handling in BooleanSimplification
## What changes were proposed in this pull request? This PR is to fix
the null handling in BooleanSimplification. In the rule
BooleanSimplification, there are two cases that do not properly handle
null values. The optimization is not right if either side is null. This
PR is to fix them.
## How was this patch tested? Added test cases
Closes #22390 from gatorsmile/fixBooleanSimplification.
Authored-by: gatorsmile <> Signed-off-by: Wenchen
Fan <>
(cherry picked from commit 79cc59718fdf7785bdc37a26bb8df4c6151114a6)
Signed-off-by: Wenchen Fan <>
(commit: db9c041790ac3a0b883d0783cbdd12ab459758c2)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala (diff)
The file was modifiedsql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/BooleanSimplificationSuite.scala (diff)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala (diff)