SuccessChanges

Summary

  1. [SPARK-26539][CORE] Remove spark.memory.useLegacyMode and (details)
  2. [SPARK-26584][SQL] Remove `spark.sql.orc.copyBatchToSpark` internal conf (details)
Commit 2f8a938805ce3c182d61bab8f66b9ff6d90dc83b by sean.owen
[SPARK-26539][CORE] Remove spark.memory.useLegacyMode and
StaticMemoryManager
## What changes were proposed in this pull request?
Remove spark.memory.useLegacyMode and StaticMemoryManager. Update tests
that used the StaticMemoryManager to equivalent use of
UnifiedMemoryManager.
## How was this patch tested?
Existing tests, with modifications to make them work with a different
mem manager.
Closes #23457 from srowen/SPARK-26539.
Authored-by: Sean Owen <sean.owen@databricks.com> Signed-off-by: Sean
Owen <sean.owen@databricks.com>
The file was modifiedcore/src/main/scala/org/apache/spark/memory/package.scala (diff)
The file was modifiedcore/src/test/scala/org/apache/spark/storage/MemoryStoreSuite.scala (diff)
The file was modifiedcore/src/main/scala/org/apache/spark/memory/UnifiedMemoryManager.scala (diff)
The file was removedcore/src/main/scala/org/apache/spark/memory/StaticMemoryManager.scala
The file was modifiedcore/src/test/scala/org/apache/spark/broadcast/BroadcastSuite.scala (diff)
The file was modifiedcore/src/test/java/org/apache/spark/memory/TaskMemoryManagerSuite.java (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/joins/HashedRelationSuite.scala (diff)
The file was modifiedcore/src/test/scala/org/apache/spark/util/collection/ExternalAppendOnlyMapSuite.scala (diff)
The file was modifiedcore/src/test/scala/org/apache/spark/util/collection/ExternalSorterSuite.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/HashedRelationMetricsBenchmark.scala (diff)
The file was modifiedcore/src/test/scala/org/apache/spark/storage/BlockManagerReplicationSuite.scala (diff)
The file was modifieddocs/configuration.md (diff)
The file was modifiedcore/src/main/scala/org/apache/spark/SparkEnv.scala (diff)
The file was modifiedstreaming/src/test/scala/org/apache/spark/streaming/ReceivedBlockHandlerSuite.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala (diff)
The file was modifiedcore/src/main/scala/org/apache/spark/SparkConf.scala (diff)
The file was removedcore/src/test/scala/org/apache/spark/memory/StaticMemoryManagerSuite.scala
The file was modifiedcore/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/AggregateBenchmark.scala (diff)
Commit 270916f8cd8ba01341f2a38a8376e9e4be08a2e8 by dongjoon
[SPARK-26584][SQL] Remove `spark.sql.orc.copyBatchToSpark` internal conf
## What changes were proposed in this pull request?
This PR aims to remove internal ORC configuration to simplify the code
path for Spark 3.0.0. This removes the configuration
`spark.sql.orc.copyBatchToSpark` and related ORC codes including tests
and benchmarks.
## How was this patch tested?
Pass the Jenkins with the reduced test coverage.
Closes #23503 from dongjoon-hyun/SPARK-26584.
Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon
Hyun <dongjoon@apache.org>
The file was modifiedsql/hive/benchmarks/OrcReadBenchmark-results.txt (diff)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala (diff)
The file was modifiedsql/core/benchmarks/DataSourceReadBenchmark-results.txt (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/DataSourceReadBenchmark.scala (diff)
The file was modifiedsql/core/src/main/java/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReader.java (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcColumnarBatchReaderSuite.scala (diff)
The file was modifiedsql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcReadBenchmark.scala (diff)