1. [SPARK-25368][SQL] Incorrect predicate pushdown returns wrong result (commit: 5ad644a4cefc20e4f198d614c59b8b0f75a228ba) (details)
Commit 5ad644a4cefc20e4f198d614c59b8b0f75a228ba by gatorsmile
[SPARK-25368][SQL] Incorrect predicate pushdown returns wrong result
How to reproduce:
```scala val df1 = spark.createDataFrame(Seq(
  (1, 1)
)).toDF("a", "b").withColumn("c", lit(null).cast("int")) val df2 =
|  a|  b|   c|  d|
|  1|  1|null|  0|
|  1|  1|null|  1|
`filter($"c".isNotNull)` was transformed to `(null <=> c#10)` before, but it is transformed to
`(c#10 = null)` since This
pr revert it to `(null <=> c#10)` to fix this issue.
unit tests
Closes #22368 from wangyum/SPARK-25368.
Authored-by: Yuming Wang <> Signed-off-by: gatorsmile
(cherry picked from commit 77c996403d5c761f0dfea64c5b1cb7480ba1d3ac)
Signed-off-by: gatorsmile <>
(commit: 5ad644a4cefc20e4f198d614c59b8b0f75a228ba)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala (diff)
The file was modifiedsql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/InferFiltersFromConstraintsSuite.scala (diff)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala (diff)