SuccessChanges

Summary

  1. [SPARK-24677][CORE] Avoid NoSuchElementException from MedianHeap (commit: 7be70e29dd92de36dbb30ce39623d588f48e4cac) (details)
Commit 7be70e29dd92de36dbb30ce39623d588f48e4cac by tgraves
[SPARK-24677][CORE] Avoid NoSuchElementException from MedianHeap
## What changes were proposed in this pull request? When speculation is
enabled, TaskSetManager#markPartitionCompleted should write successful
task duration to MedianHeap, not just increase tasksSuccessful.
Otherwise when TaskSetManager#checkSpeculatableTasks,tasksSuccessful
non-zero, but MedianHeap is empty. Then throw an exception
successfulTaskDurations.median java.util.NoSuchElementException:
MedianHeap is empty. Finally led to stopping SparkContext.
## How was this patch tested? TaskSetManagerSuite.scala unit
test:[SPARK-24677] MedianHeap should not be empty when speculation is
enabled
Author: sychen <sychen@ctrip.com>
Closes #21656 from cxzl25/fix_MedianHeap_empty.
(cherry picked from commit c8bee932cb644627c4049b5a07dd8028968572d9)
Signed-off-by: Thomas Graves <tgraves@apache.org>
(commit: 7be70e29dd92de36dbb30ce39623d588f48e4cac)
The file was modifiedcore/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala (diff)
The file was modifiedcore/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala (diff)
The file was modifiedcore/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala (diff)