FailedChanges

Summary

  1. [SPARK-25674][SQL] If the records are incremented by more than 1 at a (details)
  2. [SPARK-25598][STREAMING][BUILD][TEST-MAVEN] Remove flume connector in (details)
Commit 69f5e9cce14632a1f912c3632243a4e20b275365 by sean.owen
[SPARK-25674][SQL] If the records are incremented by more than 1 at a
time,the number of bytes might rarely ever get updated
## What changes were proposed in this pull request? If the records are
incremented by more than 1 at a time,the number of bytes might rarely
ever get updated,because it might skip over the count that is an exact
multiple of UPDATE_INPUT_METRICS_INTERVAL_RECORDS.
This PR just checks whether the increment causes the value to exceed a
higher multiple of UPDATE_INPUT_METRICS_INTERVAL_RECORDS.
## How was this patch tested? existed unit tests
Closes #22594 from 10110346/inputMetrics.
Authored-by: liuxian <liu.xian3@zte.com.cn> Signed-off-by: Sean Owen
<sean.owen@databricks.com>
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala (diff)
Commit a00181418911307725524641254439712e95445b by sean.owen
[SPARK-25598][STREAMING][BUILD][TEST-MAVEN] Remove flume connector in
Spark 3
## What changes were proposed in this pull request?
Removes all vestiges of Flume in the build, for Spark 3. I don't think
this needs Jenkins config changes.
## How was this patch tested?
Existing tests.
Closes #22692 from srowen/SPARK-25598.
Authored-by: Sean Owen <sean.owen@databricks.com> Signed-off-by: Sean
Owen <sean.owen@databricks.com>
The file was removedexternal/flume/src/test/java/org/apache/spark/streaming/flume/JavaFlumeStreamSuite.java
The file was removedexternal/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeBatchFetcher.scala
The file was modifiedstreaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala (diff)
The file was removedexternal/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/Logging.scala
The file was removedexternal/flume-sink/src/main/avro/sparkflume.avdl
The file was removedexternal/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeTestUtils.scala
The file was removedexternal/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/SparkSink.scala
The file was modifiedpython/pyspark/streaming/dstream.py (diff)
The file was removedexternal/flume-sink/pom.xml
The file was removedexternal/flume/pom.xml
The file was removedpython/pyspark/streaming/flume.py
The file was removedexternal/flume/src/main/scala/org/apache/spark/examples/FlumePollingEventCount.scala
The file was modifieddev/test-dependencies.sh (diff)
The file was removedexternal/flume/src/main/scala/org/apache/spark/streaming/flume/FlumePollingInputDStream.scala
The file was modifieddev/sbt-checkstyle (diff)
The file was removedexternal/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeUtils.scala
The file was removedexamples/src/main/python/streaming/flume_wordcount.py
The file was removedexternal/flume/src/main/scala/org/apache/spark/streaming/flume/package-info.java
The file was modifiedexternal/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/DirectKafkaInputDStream.scala (diff)
The file was modifieddev/mima (diff)
The file was modifieddev/create-release/release-build.sh (diff)
The file was removedexternal/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/SparkAvroCallbackHandler.scala
The file was removedexternal/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/SparkSinkThreadFactory.scala
The file was removedexternal/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/SparkSinkUtils.scala
The file was removedexternal/flume/src/main/scala/org/apache/spark/streaming/flume/EventTransformer.scala
The file was modifieddev/run-tests.py (diff)
The file was removedexternal/flume-sink/src/main/scala/org/apache/spark/streaming/flume/sink/TransactionProcessor.scala
The file was modifieddocs/building-spark.md (diff)
The file was modifiedpython/pyspark/streaming/tests.py (diff)
The file was removedexternal/flume/src/test/resources/log4j.properties
The file was modifieddev/sparktestsupport/modules.py (diff)
The file was modifiedproject/SparkBuild.scala (diff)
The file was removedexternal/flume-assembly/pom.xml
The file was modifieddocs/streaming-custom-receivers.md (diff)
The file was removedexternal/flume-sink/src/test/resources/log4j.properties
The file was removedexternal/flume/src/test/java/org/apache/spark/streaming/flume/JavaFlumePollingStreamSuite.java
The file was removedexternal/flume/src/test/java/org/apache/spark/streaming/LocalJavaStreamingContext.java
The file was modifiedstreaming/src/main/scala/org/apache/spark/streaming/api/java/JavaDStream.scala (diff)
The file was modifiedexternal/kafka-0-8/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala (diff)
The file was modifiedpom.xml (diff)
The file was removeddocs/streaming-flume-integration.md
The file was removedexternal/flume-sink/src/test/scala/org/apache/spark/streaming/flume/sink/SparkSinkSuite.scala
The file was removedexternal/flume/src/test/scala/org/apache/spark/streaming/TestOutputStream.scala
The file was modifiedstreaming/src/main/scala/org/apache/spark/streaming/dstream/InputDStream.scala (diff)
The file was modifiedpython/docs/pyspark.streaming.rst (diff)
The file was removedexternal/flume/src/main/scala/org/apache/spark/examples/FlumeEventCount.scala
The file was removedexternal/flume/src/main/scala/org/apache/spark/streaming/flume/package.scala
The file was removedexternal/flume/src/test/scala/org/apache/spark/streaming/flume/FlumeStreamSuite.scala
The file was removedexternal/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeInputDStream.scala
The file was removedexternal/flume/src/main/scala/org/apache/spark/streaming/flume/PollingFlumeTestUtils.scala
The file was modifieddocs/streaming-programming-guide.md (diff)
The file was removedexternal/flume/src/test/scala/org/apache/spark/streaming/flume/FlumePollingStreamSuite.scala
The file was modifieddev/scalastyle (diff)
The file was removedexternal/flume/src/main/java/org/apache/spark/examples/JavaFlumeEventCount.java
The file was modifiedstreaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala (diff)