1. [SPARK-24935][SQL][2.3] fix Hive UDAF with two aggregation buffers (commit: 85fc2f25b7260f278728b20da35bdae2b56c2bdc) (details)
  2. [SPARK-27577][MLLIB] Correct thresholds downsampled in (commit: 3c5fca1716130248b54389e2555b47f35c5a6901) (details)
Commit 85fc2f25b7260f278728b20da35bdae2b56c2bdc by dhyun
[SPARK-24935][SQL][2.3] fix Hive UDAF with two aggregation buffers
## What changes were proposed in this pull request?
backport and to 2.3.
## How was this patch tested?
existing tests
Closes #24539 from cloud-fan/backport.
Lead-authored-by: pgandhi <> Co-authored-by:
Wenchen Fan <> Signed-off-by: Dongjoon Hyun
(commit: 85fc2f25b7260f278728b20da35bdae2b56c2bdc)
The file was modifiedsql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala (diff)
The file was modifiedsql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveUDAFSuite.scala (diff)
Commit 3c5fca1716130248b54389e2555b47f35c5a6901 by sean.owen
[SPARK-27577][MLLIB] Correct thresholds downsampled in
## What changes were proposed in this pull request?
Choose the last record in chunks when calculating metrics with
downsampling in `BinaryClassificationMetrics`.
## How was this patch tested?
A new unit test is added to verify thresholds from downsampled records.
Closes #24470 from shishaochen/spark-mllib-binary-metrics.
Authored-by: Shaochen Shi <> Signed-off-by:
Sean Owen <>
(cherry picked from commit d5308cd86fff1e4bf9c24e0dd73d8d2c92737c4f)
Signed-off-by: Sean Owen <>
(commit: 3c5fca1716130248b54389e2555b47f35c5a6901)
The file was modifiedmllib/src/test/scala/org/apache/spark/mllib/evaluation/BinaryClassificationMetricsSuite.scala (diff)
The file was modifiedmllib/src/main/scala/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.scala (diff)