1. [SPARK-25700][SQL] Creates ReadSupport in only Append Mode in Data (details)
Commit 83e19d5b80fac6ea4b29d8eb561a5ad06835171b by dongjoon
[SPARK-25700][SQL] Creates ReadSupport in only Append Mode in Data
Source V2 write path
## What changes were proposed in this pull request?
This PR proposes to avoid to make a readsupport and read schema when it
writes in other save modes.
happened to create a readsupport in write path, which ended up with
reading schema from readsupport at write path.
This breaks
`spark.range(1).format("source")"non-existent-path")` case
since there's no way to read the schema from "non-existent-path".
See also See
also See also
## How was this patch tested?
Unit test and manual tests.
Closes #22688 from HyukjinKwon/append-revert-2.
Authored-by: hyukjinkwon <> Signed-off-by: Dongjoon
Hyun <>
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/sources/v2/SimpleWritableDataSource.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/sources/v2/DataSourceV2Suite.scala (diff)