SuccessChanges

Summary

  1. [SPARK-23670][SQL] Fix memory leak on SparkPlanGraphWrapper (commit: d9e1f7040092aa4aeab9bdb82f0c28a292c90609) (details)
  2. [SPARK-23553][TESTS] Tests should not assume the default value of (commit: 21b6de4592bf8c69db9135f409912b6a62f70894) (details)
Commit d9e1f7040092aa4aeab9bdb82f0c28a292c90609 by vanzin
[SPARK-23670][SQL] Fix memory leak on SparkPlanGraphWrapper
Clean up SparkPlanGraphWrapper objects from InMemoryStore together with
cleaning up SQLExecutionUIData existing unit test was extended to check
also SparkPlanGraphWrapper object count
vanzin
Author: myroslavlisniak <acnipin@gmail.com>
Closes #20813 from myroslavlisniak/master.
(cherry picked from commit c2632edebd978716dbfa7874a2fc0a8f5a4a9951)
Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>
(commit: d9e1f7040092aa4aeab9bdb82f0c28a292c90609)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusStore.scala (diff)
Commit 21b6de4592bf8c69db9135f409912b6a62f70894 by gatorsmile
[SPARK-23553][TESTS] Tests should not assume the default value of
`spark.sql.sources.default`
## What changes were proposed in this pull request?
Currently, some tests have an assumption that
`spark.sql.sources.default=parquet`. In fact, that is a correct
assumption, but that assumption makes it difficult to test new data
source format.
This PR aims to
- Improve test suites more robust and makes it easy to test new data
sources in the future.
- Test new native ORC data source with the full existing Apache Spark
test coverage.
As an example, the PR uses `spark.sql.sources.default=orc` during
reviews. The value should be `parquet` when this PR is accepted.
## How was this patch tested?
Pass the Jenkins with updated tests.
Author: Dongjoon Hyun <dongjoon@apache.org>
Closes #20705 from dongjoon-hyun/SPARK-23553.
(cherry picked from commit 5414abca4fec6a68174c34d22d071c20027e959d)
Signed-off-by: gatorsmile <gatorsmile@gmail.com>
(commit: 21b6de4592bf8c69db9135f409912b6a62f70894)
The file was modifiedsql/hive/src/test/scala/org/apache/spark/sql/hive/MetastoreDataSourcesSuite.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/columnar/InMemoryColumnarQuerySuite.scala (diff)
The file was modifiedsql/hive/src/test/scala/org/apache/spark/sql/hive/PartitionProviderCompatibilitySuite.scala (diff)
The file was modifiedpython/pyspark/sql/readwriter.py (diff)
The file was modifiedsql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala (diff)
The file was modifiedsql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala (diff)
The file was modifiedsql/hive/src/test/scala/org/apache/spark/sql/hive/PartitionedTablePerfStatsSuite.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetPartitionDiscoverySuite.scala (diff)