SuccessChanges

Summary

  1. [SPARK-25674][FOLLOW-UP] Update the stats for each ColumnarBatch (details)
Commit 0726bc56fce83c3ec30cfbb6c12dfcd68a85cd0f by sean.owen
[SPARK-25674][FOLLOW-UP] Update the stats for each ColumnarBatch
This PR is a follow-up of https://github.com/apache/spark/pull/22594 .
This alternative can avoid the unneeded computation in the hot code
path.
- For row-based scan, we keep the original way.
- For the columnar scan, we just need to update the stats after each
batch.
N/A
Closes #22731 from gatorsmile/udpateStatsFileScanRDD.
Authored-by: gatorsmile <gatorsmile@gmail.com> Signed-off-by: Wenchen
Fan <wenchen@databricks.com>
(cherry picked from commit 4cee191c04f14d7272347e4b29201763c6cfb6bf)
Signed-off-by: Sean Owen <sean.owen@databricks.com>
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileScanRDD.scala (diff)