1. [SPARK-20327][YARN] Follow up: fix resource request tests on Hadoop 3. (details)
Commit 7d425b190a91f49193eeb58c398e497ff92c6169 by irashid
[SPARK-20327][YARN] Follow up: fix resource request tests on Hadoop 3.
The test fix is to allocate a `Resource` object only after the resource
types have been initialized. Otherwise the YARN classes get in a weird
state and throw a different exception than expected, because the
resource has a different view of the registered resources.
I also removed a test for a null resource since that seems unnecessary
and made the fix more complicated.
All the other changes are just cleanup; basically simplify the tests by
defining what is being tested and deriving the resource type
registration and the SparkConf from that data, instead of having
redundant definitions in the tests.
Ran tests with Hadoop 3 (and also without it).
Closes #22751 from vanzin/SPARK-20327.fix.
Authored-by: Marcelo Vanzin <> Signed-off-by: Imran
Rashid <>
The file was modifiedresource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceRequestHelper.scala (diff)
The file was modifiedresource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceRequestHelperSuite.scala (diff)
The file was modifiedresource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceRequestTestHelper.scala (diff)
The file was modifiedresource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala (diff)