SuccessChanges

Summary

  1. [SPARK-26726] Synchronize the amount of memory used by the broadcast (commit: 94a4b465bba39dc866c2b060efd1002b55aa7ac2) (details)
  2. [SPARK-26757][GRAPHX] Return 0 for `count` on empty Edge/Vertex RDDs (commit: 537d15ca5edb0f21ce2a15b30866a53675f90382) (details)
Commit 94a4b465bba39dc866c2b060efd1002b55aa7ac2 by vanzin
[SPARK-26726] Synchronize the amount of memory used by the broadcast
variable to the UI display
…not synchronized to the UI display
## What changes were proposed in this pull request? The amount of memory
used by the broadcast variable is not synchronized to the UI display. I
added the case for BroadcastBlockId and updated the memory usage.
## How was this patch tested?
We can test this patch with unit tests.
Closes #23649 from httfighter/SPARK-26726.
Lead-authored-by: 韩田田00222924 <han.tiantian@zte.com.cn>
Co-authored-by: han.tiantian@zte.com.cn <han.tiantian@zte.com.cn>
Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>
(cherry picked from commit f4a17e916b729f9dc46e859b50a416db1e37b92e)
Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>
(commit: 94a4b465bba39dc866c2b060efd1002b55aa7ac2)
The file was modifiedcore/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala (diff)
The file was modifiedcore/src/main/scala/org/apache/spark/status/AppStatusListener.scala (diff)
Commit 537d15ca5edb0f21ce2a15b30866a53675f90382 by sean.owen
[SPARK-26757][GRAPHX] Return 0 for `count` on empty Edge/Vertex RDDs
## What changes were proposed in this pull request?
Previously a "java.lang.UnsupportedOperationException: empty collection"
exception would be thrown due to using `reduce`, rather than `fold` or
similar that can tolerate empty RDDs.
This behaviour has existed for the Vertex RDDs since it was introduced
in b30e0ae0351be1cbc0b1cf179293587b466ee026. It seems this behaviour was
inherited by the Edge RDDs via copy-paste in
ee29ef3800438501e0ff207feb00a28973fc0769.
## How was this patch tested?
Two new unit tests.
Closes #23681 from huonw/empty-graphx.
Authored-by: Huon Wilson <Huon.Wilson@data61.csiro.au> Signed-off-by:
Sean Owen <sean.owen@databricks.com>
(cherry picked from commit da526985c7574dccdcc0cca7452e2e999a5b3012)
Signed-off-by: Sean Owen <sean.owen@databricks.com>
(commit: 537d15ca5edb0f21ce2a15b30866a53675f90382)
The file was modifiedgraphx/src/test/scala/org/apache/spark/graphx/EdgeRDDSuite.scala (diff)
The file was modifiedgraphx/src/main/scala/org/apache/spark/graphx/impl/EdgeRDDImpl.scala (diff)
The file was modifiedgraphx/src/test/scala/org/apache/spark/graphx/VertexRDDSuite.scala (diff)
The file was modifiedgraphx/src/test/scala/org/apache/spark/graphx/lib/SVDPlusPlusSuite.scala (diff)
The file was modifiedgraphx/src/main/scala/org/apache/spark/graphx/lib/SVDPlusPlus.scala (diff)
The file was modifiedgraphx/src/main/scala/org/apache/spark/graphx/impl/VertexRDDImpl.scala (diff)