1. [SPARK-27798][SQL][BRANCH-2.3] ConvertToLocalRelation should tolerate (details)
Commit 9c9102c395372c061ed9323f2d123e519e1cfc8f by dhyun
[SPARK-27798][SQL][BRANCH-2.3] ConvertToLocalRelation should tolerate
expression reusing output object
## What changes were proposed in this pull request?
The original issue SPARK-27798 was reported on master branch. When using
`from_avro` to deserialize avro data to catalyst StructType format, if
`ConvertToLocalRelation` is applied at the time, `from_avro` produces
only the last value (overriding previous values).
The cause is `AvroDeserializer` reuses output row for StructType. But
`ConvertToLocalRelation` doesn't tolerate expression reusing output
This is to backport the fix of `ConvertToLocalRelation` to branch 2.3.
## How was this patch tested?
Added test.
Closes #24823 from viirya/SPARK-27798-2.3.
Authored-by: Liang-Chi Hsieh <> Signed-off-by: Dongjoon
Hyun <>
The file was modifiedsql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/ConvertToLocalRelationSuite.scala (diff)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala (diff)