1. [SPARK-23264][SQL] Fix scala.MatchError in literals.sql.out (commit: 1e552b356253bfbb2bdf8156c15630d5ed65dfc6) (details)
  2. [SPARK-23288][SS] Fix output metrics with parquet sink (commit: 4b9f33ff7f5268ffcc11cd5eea5976e9153be9fb) (details)
  3. [SPARK-23729][CORE] Respect URI fragment when resolving globs (commit: c9acd46bed8fa3e410e8a44aafe3237e59deaa73) (details)
Commit 1e552b356253bfbb2bdf8156c15630d5ed65dfc6 by wenchen
[SPARK-23264][SQL] Fix scala.MatchError in literals.sql.out
## What changes were proposed in this pull request? To fix
`scala.MatchError` in `literals.sql.out`, this pr added an entry for
`CalendarIntervalType` in `QueryExecution.toHiveStructString`.
## How was this patch tested? Existing tests and added tests in
Author: Takeshi Yamamuro <>
Closes #20872 from maropu/FixIntervalTests.
(cherry picked from commit 98d0ea3f6091730285293321a50148f69e94c9cd)
Signed-off-by: Wenchen Fan <>
(commit: 1e552b356253bfbb2bdf8156c15630d5ed65dfc6)
The file was modifiedsql/core/src/test/resources/sql-tests/inputs/literals.sql (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala (diff)
The file was modifiedsql/core/src/test/resources/sql-tests/results/literals.sql.out (diff)
Commit 4b9f33ff7f5268ffcc11cd5eea5976e9153be9fb by wenchen
[SPARK-23288][SS] Fix output metrics with parquet sink
## What changes were proposed in this pull request?
Output metrics were not filled when parquet sink used.
This PR fixes this problem by passing a `BasicWriteJobStatsTracker` in
## How was this patch tested?
Additional unit test added.
Author: Gabor Somogyi <>
Closes #20745 from gaborgsomogyi/SPARK-23288.
(cherry picked from commit 918c7e99afdcea05c36626e230636c4f8aabf82c)
Signed-off-by: Wenchen Fan <>
(commit: 4b9f33ff7f5268ffcc11cd5eea5976e9153be9fb)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSinkSuite.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/command/DataWritingCommand.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSink.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/datasources/BasicWriteStatsTracker.scala (diff)
Commit c9acd46bed8fa3e410e8a44aafe3237e59deaa73 by vanzin
[SPARK-23729][CORE] Respect URI fragment when resolving globs
Firstly, glob resolution will not result in swallowing the remote name
part (that is preceded by the `#` sign) in case of `--files` or
`--archives` options
Moreover in the special case of multiple resolutions when the remote
naming does not make sense and error is returned.
Enhanced current test and wrote additional test for the error case
Author: Mihaly Toth <>
Closes #20853 from misutoth/glob-with-remote-name.
(cherry picked from commit 0604beaff2baa2d0fed86c0c87fd2a16a1838b5f)
Signed-off-by: Marcelo Vanzin <>
(commit: c9acd46bed8fa3e410e8a44aafe3237e59deaa73)
The file was modifiedcore/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala (diff)
The file was modifiedcore/src/main/scala/org/apache/spark/deploy/DependencyUtils.scala (diff)
The file was modifiedcore/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala (diff)