1. [SPARK-25521][SQL] Job id showing null in the logs when insert into (details)
  2. [SPARK-24601] Update Jackson to 2.9.6 (details)
Commit 459700727fadf3f35a211eab2ffc8d68a4a1c39a by wenchen
[SPARK-25521][SQL] Job id showing null in the logs when insert into
command Job is finished.
## What changes were proposed in this pull request?
``As part of  insert command  in FileFormatWriter, a job context is
created for handling the write operation , While initializing the job
context using setupJob() API in HadoopMapReduceCommitProtocol , we set
the jobid  in the Jobcontext configuration.In FileFormatWriter since we
are directly getting the jobId from the map reduce JobContext the job id
will come as null  while adding the log. As a solution we shall get the
jobID from the configuration of the map reduce Jobcontext.``
## How was this patch tested? Manually, verified the logs after the
Closes #22572 from sujith71955/master_log_issue.
Authored-by: s71955 <> Signed-off-by: Wenchen
Fan <>
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala (diff)
Commit ab1650d2938db4901b8c28df945d6a0691a19d31 by hyukjinkwon
[SPARK-24601] Update Jackson to 2.9.6
Hi all,
Jackson is incompatible with upstream versions, therefore bump the
Jackson version to a more recent one. I bumped into some issues with
Azure CosmosDB that is using a more recent version of Jackson. This can
be fixed by adding exclusions and then it works without any issues. So
no breaking changes in the API's.
I would also consider bumping the version of Jackson in Spark. I would
suggest to keep up to date with the dependencies, since in the future
this issue will pop up more frequently.
## What changes were proposed in this pull request?
Bump Jackson to 2.9.6
## How was this patch tested?
Compiled and tested it locally to see if anything broke.
Please review before opening a
pull request.
Closes #21596 from Fokko/fd-bump-jackson.
Authored-by: Fokko Driesprong <>
Signed-off-by: hyukjinkwon <>
The file was modifiedcore/src/main/scala/org/apache/spark/status/api/v1/api.scala (diff)
The file was modifiedpom.xml (diff)
The file was modifieddev/deps/spark-deps-hadoop-3.1 (diff)
The file was modifiedcore/src/main/scala/org/apache/spark/status/KVUtils.scala (diff)
The file was modifiedcore/src/main/scala/org/apache/spark/rdd/RDDOperationScope.scala (diff)
The file was modifiedsql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/JsonExpressionsSuite.scala (diff)
The file was modifiedcore/src/main/scala/org/apache/spark/deploy/rest/SubmitRestProtocolMessage.scala (diff)
The file was modifiedcore/src/main/scala/org/apache/spark/status/api/v1/JacksonMessageWriter.scala (diff)
The file was modifieddev/deps/spark-deps-hadoop-2.7 (diff)
The file was modifieddev/deps/spark-deps-hadoop-2.6 (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonBenchmarks.scala (diff)