1. [SPARK-24755][CORE] Executor loss can cause task to not be resubmitted (commit: d0280ab818391fd11662647459f1e9e683b2bc8e) (details)
Commit d0280ab818391fd11662647459f1e9e683b2bc8e by tgraves
[SPARK-24755][CORE] Executor loss can cause task to not be resubmitted
**Description** As described in
[SPARK-24755](, when
speculation is enabled, there is scenario that executor loss can cause
task to not be resubmitted. This patch changes the variable
killedByOtherAttempt to keeps track of the taskId of tasks that are
killed by other attempt. By doing this, we can still prevent
resubmitting task killed by other attempt while resubmit successful
attempt when executor lost.
**How was this patch tested?** A UT is added based on the UT written by
xuanyuanking with modification to simulate the scenario described in
Author: Hieu Huynh <“”>
Closes #21729 from hthuynh2/SPARK_24755.
(cherry picked from commit 8d707b06003bc97d06630b22e6ae7c35f99b3cdd)
Signed-off-by: Thomas Graves <>
(commit: d0280ab818391fd11662647459f1e9e683b2bc8e)
The file was modifiedcore/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala (diff)
The file was modifiedcore/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala (diff)