FailedChanges

Summary

  1. [SPARK-25479][TEST] Refactor DatasetBenchmark to use main method (details)
  2. [SPARK-17159][STREAM] Significant speed up for running spark streaming (details)
Commit 95ae2094618fbbe07008c190105053dc2b85da1a by dongjoon
[SPARK-25479][TEST] Refactor DatasetBenchmark to use main method
## What changes were proposed in this pull request?
Refactor `DatasetBenchmark` to use main method. Generate benchmark
result:
```sh SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "sql/test:runMain
org.apache.spark.sql.DatasetBenchmark"
```
## How was this patch tested?
manual tests
Closes #22488 from wangyum/SPARK-25479.
Lead-authored-by: Yuming Wang <yumwang@ebay.com> Co-authored-by:
Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun
<dongjoon@apache.org>
The file was addedsql/core/benchmarks/DatasetBenchmark-results.txt
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/DatasetBenchmark.scala (diff)
Commit 3ae4f07de06e267f0363a53264876ea99dd731df by sean.owen
[SPARK-17159][STREAM] Significant speed up for running spark streaming
against Object store.
## What changes were proposed in this pull request?
Original work by Steve Loughran. Based on #17745.
This is a minimal patch of changes to FileInputDStream to reduce File
status requests when querying files. Each call to file status is 3+ http
calls to object store. This patch eliminates the need for it, by using
FileStatus objects.
This is a minor optimisation when working with filesystems, but
significant when working with object stores.
## How was this patch tested?
Tests included. Existing tests pass.
Closes #22339 from ScrapCodes/PR_17745.
Lead-authored-by: Prashant Sharma <prashant@apache.org> Co-authored-by:
Steve Loughran <stevel@hortonworks.com> Signed-off-by: Sean Owen
<sean.owen@databricks.com>
The file was modifiedstreaming/src/test/scala/org/apache/spark/streaming/TestSuiteBase.scala (diff)
The file was modifiedstreaming/src/test/scala/org/apache/spark/streaming/InputStreamsSuite.scala (diff)
The file was modifiedstreaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala (diff)