1. [SPARK-24257][SQL] LongToUnsafeRowMap calculate the new size may be (details)
Commit 5fd080902515455f9ce8d79b7853c5739d3847e6 by wenchen
[SPARK-24257][SQL] LongToUnsafeRowMap calculate the new size may be
LongToUnsafeRowMap has a mistake when growing its page array: it blindly
grows to `oldSize * 2`, while the new record may be larger than `oldSize
* 2`. Then we may have a malformed UnsafeRow when querying this map,
whose actual data is smaller than its declared size, and the data is
Author: sychen <>
Closes #21311 from cxzl25/fix_LongToUnsafeRowMap_page_size.
(cherry picked from commit 888340151f737bb68d0e419b1e949f11469881f9)
Signed-off-by: Wenchen Fan <>
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/joins/HashedRelationSuite.scala (diff)