SuccessChanges

Summary

  1. [SPARK-25408] Move to more ideomatic Java8 (details)
  2. [SPARK-25641] Change the (details)
Commit 1a28625355d75076bde4bcc95a72e9b187cda606 by sean.owen
[SPARK-25408] Move to more ideomatic Java8
While working on another PR, I noticed that there is quite some legacy
Java in there that can be beautified. For example the use of features
from Java8, such as:
- Collection libraries
- Try-with-resource blocks
No logic has been changed. I think it is important to have a solid
codebase with examples that will inspire next PR's to follow up on the
best practices.
What are your thoughts on this?
This makes code easier to read, and using try-with-resource makes is
less likely to forget to close something.
## What changes were proposed in this pull request?
No changes in the logic of Spark, but more in the aesthetics of the
code.
## How was this patch tested?
Using the existing unit tests. Since no logic is changed, the existing
unit tests should pass.
Please review http://spark.apache.org/contributing.html before opening a
pull request.
Closes #22637 from Fokko/SPARK-25408.
Authored-by: Fokko Driesprong <fokkodriesprong@godatadriven.com>
Signed-off-by: Sean Owen <sean.owen@databricks.com>
The file was modifiedcommon/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketchImpl.java (diff)
The file was modifiedcommon/sketch/src/main/java/org/apache/spark/util/sketch/CountMinSketch.java (diff)
The file was modifiedcore/src/main/java/org/apache/spark/shuffle/sort/BypassMergeSortShuffleWriter.java (diff)
The file was modifiedcommon/kvstore/src/test/java/org/apache/spark/util/kvstore/LevelDBSuite.java (diff)
The file was modifiedcore/src/main/java/org/apache/spark/io/ReadAheadInputStream.java (diff)
The file was modifiedcommon/kvstore/src/main/java/org/apache/spark/util/kvstore/KVStoreSerializer.java (diff)
The file was modifiedcore/src/test/java/org/apache/spark/JavaJdbcRDDSuite.java (diff)
The file was modifiedsql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/RowBasedKeyValueBatch.java (diff)
The file was modifiedcommon/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolverSuite.java (diff)
The file was modifiedcore/src/test/java/org/apache/spark/shuffle/sort/UnsafeShuffleWriterSuite.java (diff)
The file was modifiedsql/catalyst/src/test/java/org/apache/spark/sql/catalyst/expressions/RowBasedKeyValueBatchSuite.java (diff)
The file was modifiedcommon/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleSecuritySuite.java (diff)
The file was modifiedcommon/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ShuffleIndexInformation.java (diff)
The file was modifiedcore/src/test/java/test/org/apache/spark/JavaAPISuite.java (diff)
The file was modifiedcommon/network-common/src/test/java/org/apache/spark/network/ChunkFetchIntegrationSuite.java (diff)
The file was modifiedcommon/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleIntegrationSuite.java (diff)
The file was modifiedcore/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java (diff)
Commit 6353425af76f9cc9de7ee4094f41df7a7390d898 by tgraves
[SPARK-25641] Change the
spark.shuffle.server.chunkFetchHandlerThreadsPercent default to 100
## What changes were proposed in this pull request?
We want to change the default percentage to 100 for
spark.shuffle.server.chunkFetchHandlerThreadsPercent. The reason being
currently this is set to 0. Which means currently if server.ioThreads >
0, the default number of threads would be 2 * #cores instead of
server.io.Threads. We want the default to server.io.Threads in case this
is not set at all. Also here a default of 0 would also mean 2 * #cores
## How was this patch tested? Manual
(Please explain how this patch was tested. E.g. unit tests, integration
tests, manual tests)
(If this patch involves UI changes, please attach a screenshot;
otherwise, remove this)
Please review http://spark.apache.org/contributing.html before opening a
pull request.
Closes #22628 from redsanket/SPARK-25641.
Lead-authored-by: Sanket Chintapalli <schintap@yahoo-inc.com>
Co-authored-by: Sanket Chintapalli <chintapalli.sanketreddy@gmail.com>
Signed-off-by: Thomas Graves <tgraves@apache.org>
The file was modifiedcommon/network-common/src/main/java/org/apache/spark/network/util/TransportConf.java (diff)