FailedChanges

Summary

  1. [SPARK-24067][BACKPORT-2.3][STREAMING][KAFKA] Allow non-consecutive (commit: 1d598b771de3b588a2f377ae7ccf8193156641f2) (details)
  2. [SPARKR] Require Java 8 for SparkR (commit: 7de4bef9ec37440aa36e6b0e9d8656de07d03b68) (details)
Commit 1d598b771de3b588a2f377ae7ccf8193156641f2 by cody
[SPARK-24067][BACKPORT-2.3][STREAMING][KAFKA] Allow non-consecutive
offsets
## What changes were proposed in this pull request?
Backport of the bugfix in SPARK-17147
Add a configuration spark.streaming.kafka.allowNonConsecutiveOffsets to
allow streaming jobs to proceed on compacted topics (or other situations
involving gaps between offsets in the log).
## How was this patch tested?
Added new unit test
justinrmiller has been testing this branch in production for a few weeks
Author: cody koeninger <cody@koeninger.org>
Closes #21300 from koeninger/branch-2.3_kafkafix.
(commit: 1d598b771de3b588a2f377ae7ccf8193156641f2)
The file was modifiedexternal/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaRDD.scala (diff)
The file was modifiedexternal/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaRDDSuite.scala (diff)
The file was addedexternal/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/mocks/MockTime.scala
The file was modifiedexternal/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaTestUtils.scala (diff)
The file was modifiedexternal/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/CachedKafkaConsumer.scala (diff)
The file was addedexternal/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/mocks/MockScheduler.scala
Commit 7de4bef9ec37440aa36e6b0e9d8656de07d03b68 by shivaram
[SPARKR] Require Java 8 for SparkR
This change updates the SystemRequirements and also includes a runtime
check if the JVM is being launched by R. The runtime check is done by
querying `java -version`
## How was this patch tested?
Tested on a Mac and Windows machine
Author: Shivaram Venkataraman <shivaram@cs.berkeley.edu>
Closes #21278 from shivaram/sparkr-skip-solaris.
(cherry picked from commit f27a035daf705766d3445e5c6a99867c11c552b0)
Signed-off-by: Shivaram Venkataraman <shivaram@cs.berkeley.edu>
(commit: 7de4bef9ec37440aa36e6b0e9d8656de07d03b68)
The file was modifiedR/pkg/R/client.R (diff)
The file was modifiedR/pkg/R/sparkR.R (diff)
The file was modifiedR/pkg/R/utils.R (diff)
The file was modifiedR/pkg/DESCRIPTION (diff)