1. [SPARK-32428][EXAMPLES] Make BinaryClassificationMetricsExample cons… (details)
Commit 62671af4160306f8f007fef0628b2a77da9b2824 by srowen
[SPARK-32428][EXAMPLES] Make BinaryClassificationMetricsExample cons…
…istently print the metrics on driver's stdout
### What changes were proposed in this pull request?
Call collect on RDD before calling foreach so that it sends the result
to the driver node and print it on this node's stdout.
### Why are the changes needed?
Some RDDs in this example (e.g., precision, recall) call println without
calling collect. If the job is under local mode, it sends the data to
the driver node and prints the metrics on the driver's stdout. However
if the job is under cluster mode, the job prints the metrics on the
executor's stdout. It seems inconsistent compared to the other metrics
nothing to do with RDD (e.g., auPRC, auROC) since these metrics always
output the result on the driver's stdout. All of the metrics should
output its result on the driver's stdout.
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
This is example code. It doesn't have any tests.
Closes #29222 from titsuki/SPARK-32428.
Authored-by: Itsuki Toyota <> Signed-off-by: Sean Owen
(cherry picked from commit 86ead044e3789b3291a38ec2142cbb343d1290c1)
Signed-off-by: Sean Owen <>
The file was modifiedexamples/src/main/scala/org/apache/spark/examples/mllib/ChiSqSelectorExample.scala (diff)
The file was modifiedexamples/src/main/scala/org/apache/spark/examples/mllib/ElementwiseProductExample.scala (diff)
The file was modifiedexamples/src/main/scala/org/apache/spark/examples/mllib/BinaryClassificationMetricsExample.scala (diff)
The file was modifiedexamples/src/main/scala/org/apache/spark/examples/mllib/StandardScalerExample.scala (diff)
The file was modifiedexamples/src/main/scala/org/apache/spark/examples/mllib/TFIDFExample.scala (diff)
The file was modifiedexamples/src/main/scala/org/apache/spark/examples/mllib/NormalizerExample.scala (diff)