SuccessChanges

Summary

  1. [SPARK-23732][DOCS] Fix source links in generated scaladoc. (commit: 4d4548a6eba0fb3e05c6e2457a51541475e456d3) (details)
  2. [SPARK-24502][SQL] flaky test: UnsafeRowSerializerSuite (commit: bf58687572738ffc1a74bee9cffa310d4674ce31) (details)
  3. [SPARK-24531][TESTS] Remove version 2.2.0 from testing versions in (commit: 63e1da16294e02affa27ab43a3ef0ae62e0c7672) (details)
  4. [SPARK-24506][UI] Add UI filters to tabs added after binding (commit: a55de387db901975e68e88c68ec464d49f240270) (details)
Commit 4d4548a6eba0fb3e05c6e2457a51541475e456d3 by hyukjinkwon
[SPARK-23732][DOCS] Fix source links in generated scaladoc.
Apply the suggestion on the bug to fix source links. Tested with the
2.3.1 release docs.
Author: Marcelo Vanzin <vanzin@cloudera.com>
Closes #21521 from vanzin/SPARK-23732.
(commit: 4d4548a6eba0fb3e05c6e2457a51541475e456d3)
The file was modifiedproject/SparkBuild.scala (diff)
Commit bf58687572738ffc1a74bee9cffa310d4674ce31 by wenchen
[SPARK-24502][SQL] flaky test: UnsafeRowSerializerSuite
## What changes were proposed in this pull request?
`UnsafeRowSerializerSuite` calls `UnsafeProjection.create` which
accesses `SQLConf.get`, while the current active SparkSession may
already be stopped, and we may hit exception like this
``` sbt.ForkMain$ForkError: java.lang.IllegalStateException:
LiveListenerBus is stopped.
at
org.apache.spark.scheduler.LiveListenerBus.addToQueue(LiveListenerBus.scala:97)
at
org.apache.spark.scheduler.LiveListenerBus.addToStatusQueue(LiveListenerBus.scala:80)
at
org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:93)
at
org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:120)
at
org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:120)
at scala.Option.getOrElse(Option.scala:121)
at
org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:120)
at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:119)
at
org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:286)
at
org.apache.spark.sql.test.TestSparkSession.sessionState$lzycompute(TestSQLContext.scala:42)
at
org.apache.spark.sql.test.TestSparkSession.sessionState(TestSQLContext.scala:41)
at
org.apache.spark.sql.SparkSession$$anonfun$1$$anonfun$apply$1.apply(SparkSession.scala:95)
at
org.apache.spark.sql.SparkSession$$anonfun$1$$anonfun$apply$1.apply(SparkSession.scala:95)
at scala.Option.map(Option.scala:146)
at
org.apache.spark.sql.SparkSession$$anonfun$1.apply(SparkSession.scala:95)
at
org.apache.spark.sql.SparkSession$$anonfun$1.apply(SparkSession.scala:94)
at org.apache.spark.sql.internal.SQLConf$.get(SQLConf.scala:126)
at
org.apache.spark.sql.catalyst.expressions.CodeGeneratorWithInterpretedFallback.createObject(CodeGeneratorWithInterpretedFallback.scala:54)
at
org.apache.spark.sql.catalyst.expressions.UnsafeProjection$.create(Projection.scala:157)
at
org.apache.spark.sql.catalyst.expressions.UnsafeProjection$.create(Projection.scala:150)
at
org.apache.spark.sql.execution.UnsafeRowSerializerSuite.org$apache$spark$sql$execution$UnsafeRowSerializerSuite$$unsafeRowConverter(UnsafeRowSerializerSuite.scala:54)
at
org.apache.spark.sql.execution.UnsafeRowSerializerSuite.org$apache$spark$sql$execution$UnsafeRowSerializerSuite$$toUnsafeRow(UnsafeRowSerializerSuite.scala:49)
at
org.apache.spark.sql.execution.UnsafeRowSerializerSuite$$anonfun$2.apply(UnsafeRowSerializerSuite.scala:63)
at
org.apache.spark.sql.execution.UnsafeRowSerializerSuite$$anonfun$2.apply(UnsafeRowSerializerSuite.scala:60)
...
```
## How was this patch tested?
N/A
Author: Wenchen Fan <wenchen@databricks.com>
Closes #21518 from cloud-fan/test.
(cherry picked from commit 01452ea9c75ff027ceeb8314368c6bbedefdb2bf)
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
(commit: bf58687572738ffc1a74bee9cffa310d4674ce31)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/LocalSparkSession.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/UnsafeRowSerializerSuite.scala (diff)
Commit 63e1da16294e02affa27ab43a3ef0ae62e0c7672 by gatorsmile
[SPARK-24531][TESTS] Remove version 2.2.0 from testing versions in
HiveExternalCatalogVersionsSuite
## What changes were proposed in this pull request?
Removing version 2.2.0 from testing versions in
HiveExternalCatalogVersionsSuite as it is not present anymore in the
mirrors and this is blocking all the open PRs.
## How was this patch tested?
running UTs
Author: Marco Gaido <marcogaido91@gmail.com>
Closes #21540 from mgaido91/SPARK-24531.
(cherry picked from commit 2824f1436bb0371b7216730455f02456ef8479ce)
Signed-off-by: Xiao Li <gatorsmile@gmail.com>
(commit: 63e1da16294e02affa27ab43a3ef0ae62e0c7672)
The file was modifiedsql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala (diff)
Commit a55de387db901975e68e88c68ec464d49f240270 by vanzin
[SPARK-24506][UI] Add UI filters to tabs added after binding
## What changes were proposed in this pull request?
Currently, `spark.ui.filters` are not applied to the handlers added
after binding the server. This means that every page which is added
after starting the UI will not have the filters configured on it. This
can allow unauthorized access to the pages.
The PR adds the filters also to the handlers added after the UI starts.
## How was this patch tested?
manual tests (without the patch, starting the thriftserver with `--conf
spark.ui.filters=org.apache.hadoop.security.authentication.server.AuthenticationFilter
--conf
spark.org.apache.hadoop.security.authentication.server.AuthenticationFilter.params="type=simple"`
you can access `http://localhost:4040/sqlserver`; with the patch, 401 is
the response as for the other pages).
Author: Marco Gaido <marcogaido91@gmail.com>
Closes #21523 from mgaido91/SPARK-24506.
(cherry picked from commit f53818d35bdef5d20a2718b14a2fed4c468545c6)
Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>
(commit: a55de387db901975e68e88c68ec464d49f240270)
The file was modifiedcore/src/main/scala/org/apache/spark/deploy/history/HistoryServer.scala (diff)
The file was modifiedcore/src/main/scala/org/apache/spark/ui/JettyUtils.scala (diff)