FailedChanges

Summary

  1. [SPARK-25921][FOLLOW UP][PYSPARK] Fix barrier task run without (details)
  2. [SPARK-26580][SQL] remove Scala 2.11 hack for Scala UDF (details)
Commit 98e831d32115aeec4ded235bf42978f407e376da by gurwls223
[SPARK-25921][FOLLOW UP][PYSPARK] Fix barrier task run without
BarrierTaskContext while python worker reuse
## What changes were proposed in this pull request?
It's the follow-up PR for #22962, contains the following works:
- Remove `__init__` in TaskContext and BarrierTaskContext.
- Add more comments to explain the fix.
- Rewrite UT in a new class.
## How was this patch tested?
New UT in test_taskcontext.py
Closes #23435 from xuanyuanking/SPARK-25921-follow.
Authored-by: Yuanjian Li <xyliyuanjian@gmail.com> Signed-off-by: Hyukjin
Kwon <gurwls223@apache.org>
The file was modifiedpython/pyspark/tests/test_taskcontext.py (diff)
The file was modifiedpython/pyspark/taskcontext.py (diff)
Commit 1f1d98c6facd556b70f457184231b5af78de8d53 by wenchen
[SPARK-26580][SQL] remove Scala 2.11 hack for Scala UDF
## What changes were proposed in this pull request?
In https://github.com/apache/spark/pull/22732 , we tried our best to
keep the behavior of Scala UDF unchanged in Spark 2.4.
However, since Spark 3.0, Scala 2.12 is the default. The trick that was
used to keep the behavior unchanged doesn't work with Scala 2.12.
This PR proposes to remove the Scala 2.11 hack, as it's not useful.
## How was this patch tested?
existing tests.
Closes #23498 from cloud-fan/udf.
Authored-by: Wenchen Fan <wenchen@databricks.com> Signed-off-by: Wenchen
Fan <wenchen@databricks.com>
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/expressions/UserDefinedFunction.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/functions.scala (diff)
The file was modifieddocs/sql-migration-guide-upgrade.md (diff)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/UDFSuite.scala (diff)