SuccessChanges

Summary

  1. [SPARK-25425][SQL][BACKPORT-2.3] Extra options should override session (commit: cbb228e48bb046e7d88d6bf1c9b9e3b252241552) (details)
  2. [SPARK-25509][CORE] Windows doesn't support POSIX permissions (commit: 2381d60a279002e58f6c74d3d39f31b344ceff46) (details)
Commit cbb228e48bb046e7d88d6bf1c9b9e3b252241552 by dongjoon
[SPARK-25425][SQL][BACKPORT-2.3] Extra options should override session
options in DataSource V2
## What changes were proposed in this pull request?
In the PR, I propose overriding session options by extra options in
DataSource V2. Extra options are more specific and set via `.option()`,
and should overwrite more generic session options.
## How was this patch tested?
Added tests for read and write paths.
Closes #22489 from MaxGekk/session-options-2.3.
Authored-by: Maxim Gekk <maxim.gekk@databricks.com> Signed-off-by:
Dongjoon Hyun <dongjoon@apache.org>
(commit: cbb228e48bb046e7d88d6bf1c9b9e3b252241552)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/sources/v2/SimpleWritableDataSource.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/sources/v2/DataSourceV2Suite.scala (diff)
Commit 2381d60a279002e58f6c74d3d39f31b344ceff46 by sean.owen
[SPARK-25509][CORE] Windows doesn't support POSIX permissions
SHS V2 cannot enabled in Windows, because windows doesn't support POSIX
permission.
test case fails in windows without this fix.
org.apache.spark.deploy.history.HistoryServerDiskManagerSuite
test("leasing space")
SHS V2  cannot run successfully in Windows without this fix.
java.lang.UnsupportedOperationException: 'posix:permissions' not
supported as initial attribute
at
sun.nio.fs.WindowsSecurityDescriptor.fromAttribute(WindowsSecurityDescriptor.java:358)
Closes #22520 from jianjianjiao/FixWindowsPermssionsIssue.
Authored-by: Rong Tang <rotang@microsoft.com> Signed-off-by: Sean Owen
<sean.owen@databricks.com>
(cherry picked from commit a2ac5a72ccd2b14c8492d4a6da9e8b30f0f3c9b4)
Signed-off-by: Sean Owen <sean.owen@databricks.com>
(commit: 2381d60a279002e58f6c74d3d39f31b344ceff46)
The file was modifiedcore/src/main/scala/org/apache/spark/deploy/history/HistoryServerDiskManager.scala (diff)