1. [SPARK-25714] Fix Null Handling in the Optimizer rule (details)
Commit c9ba59d38e2be17b802156b49d374a726e66c6b9 by gatorsmile
[SPARK-25714] Fix Null Handling in the Optimizer rule
## What changes were proposed in this pull request?
   val df1 = Seq(("abc", 1), (null, 3)).toDF("col1", "col2")
   val df2 ="/tmp/test1")
   df2.filter("col1 = 'abc' OR (col1 != 'abc' AND col2 == 3)").show()
Before the PR, it returns both rows. After the fix, it returns `Row
("abc", 1))`. This is to fix the bug in NULL handling in
BooleanSimplification. This is a bug introduced in Spark 1.6 release.
## How was this patch tested? Added test cases
Closes #22702 from gatorsmile/fixBooleanSimplify2.
Authored-by: gatorsmile <> Signed-off-by: gatorsmile
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala (diff)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala (diff)
The file was modifiedsql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/BooleanSimplificationSuite.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala (diff)