1. [SPARK-24891][SQL] Fix HandleNullInputsForUDF rule (commit: 6a59992866a971abf6052e479ba48c6abded4d04) (details)
  2. [SPARK-24891][FOLLOWUP][HOT-FIX][2.3] Fix the Compilation Errors (commit: 740606eb8714c2536f239b454b0d0f5790886414) (details)
Commit 6a59992866a971abf6052e479ba48c6abded4d04 by gatorsmile
[SPARK-24891][SQL] Fix HandleNullInputsForUDF rule
The HandleNullInputsForUDF would always add a new `If` node every time
it is applied. That would cause a difference between the same plan being
analyzed once and being analyzed twice (or more), thus raising issues
like plan not matched in the cache manager. The solution is to mark the
arguments as null-checked, which is to add a "KnownNotNull" node above
those arguments, when adding the UDF under an `If` node, because clearly
the UDF will not be called when any of those arguments is null.
Add new tests under sql/UDFSuite and AnalysisSuite.
Author: maryannxue <>
Closes #21851 from maryannxue/spark-24891.
(commit: 6a59992866a971abf6052e479ba48c6abded4d04)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/UDFSuite.scala (diff)
The file was modifiedsql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala (diff)
The file was addedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/constraintExpressions.scala
Commit 740606eb8714c2536f239b454b0d0f5790886414 by hyukjinkwon
[SPARK-24891][FOLLOWUP][HOT-FIX][2.3] Fix the Compilation Errors
## What changes were proposed in this pull request?
This PR is to fix the compilation failure in 2.3 build.
## How was this patch tested? N/A
Author: Xiao Li <>
Closes #21869 from gatorsmile/testSPARK-24891.
(commit: 740606eb8714c2536f239b454b0d0f5790886414)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/constraintExpressions.scala (diff)