SuccessChanges

Summary

  1. [SPARK-22144][SQL] ExchangeCoordinator combine the partitions of an 0 (commit: 048197749ef990e4def1fcbf488f3ded38d95cae) (details)
  2. [SPARK-23732][DOCS] Fix source links in generated scaladoc. (commit: dc22465f3e1ef5ad59306b1f591d6fd16d674eb7) (details)
  3. [SPARK-24502][SQL] flaky test: UnsafeRowSerializerSuite (commit: 01452ea9c75ff027ceeb8314368c6bbedefdb2bf) (details)
  4. docs: fix typo (commit: 1d7db65e968de1c601e7f8b1ec9bc783ef2dbd01) (details)
  5. [SPARK-15064][ML] Locale support in StopWordsRemover (commit: 5d6a53d9831cc1e2115560db5cebe0eea2565dcd) (details)
  6. [SPARK-24531][TESTS] Remove version 2.2.0 from testing versions in (commit: 2824f1436bb0371b7216730455f02456ef8479ce) (details)
Commit 048197749ef990e4def1fcbf488f3ded38d95cae by wenchen
[SPARK-22144][SQL] ExchangeCoordinator combine the partitions of an 0
sized pre-shuffle to 0
## What changes were proposed in this pull request? when the length of
pre-shuffle's partitions is 0, the length of post-shuffle's partitions
should be 0 instead of spark.sql.shuffle.partitions.
## How was this patch tested? ExchangeCoordinator converted a
pre-shuffle that partitions is 0 to a post-shuffle that partitions is 0
instead of one that partitions is spark.sql.shuffle.partitions.
Author: liutang123 <liutang123@yeah.net>
Closes #19364 from liutang123/SPARK-22144.
(commit: 048197749ef990e4def1fcbf488f3ded38d95cae)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ExchangeCoordinator.scala (diff)
Commit dc22465f3e1ef5ad59306b1f591d6fd16d674eb7 by hyukjinkwon
[SPARK-23732][DOCS] Fix source links in generated scaladoc.
Apply the suggestion on the bug to fix source links. Tested with the
2.3.1 release docs.
Author: Marcelo Vanzin <vanzin@cloudera.com>
Closes #21521 from vanzin/SPARK-23732.
(commit: dc22465f3e1ef5ad59306b1f591d6fd16d674eb7)
The file was modifiedproject/SparkBuild.scala (diff)
Commit 01452ea9c75ff027ceeb8314368c6bbedefdb2bf by wenchen
[SPARK-24502][SQL] flaky test: UnsafeRowSerializerSuite
## What changes were proposed in this pull request?
`UnsafeRowSerializerSuite` calls `UnsafeProjection.create` which
accesses `SQLConf.get`, while the current active SparkSession may
already be stopped, and we may hit exception like this
``` sbt.ForkMain$ForkError: java.lang.IllegalStateException:
LiveListenerBus is stopped.
at
org.apache.spark.scheduler.LiveListenerBus.addToQueue(LiveListenerBus.scala:97)
at
org.apache.spark.scheduler.LiveListenerBus.addToStatusQueue(LiveListenerBus.scala:80)
at
org.apache.spark.sql.internal.SharedState.<init>(SharedState.scala:93)
at
org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:120)
at
org.apache.spark.sql.SparkSession$$anonfun$sharedState$1.apply(SparkSession.scala:120)
at scala.Option.getOrElse(Option.scala:121)
at
org.apache.spark.sql.SparkSession.sharedState$lzycompute(SparkSession.scala:120)
at org.apache.spark.sql.SparkSession.sharedState(SparkSession.scala:119)
at
org.apache.spark.sql.internal.BaseSessionStateBuilder.build(BaseSessionStateBuilder.scala:286)
at
org.apache.spark.sql.test.TestSparkSession.sessionState$lzycompute(TestSQLContext.scala:42)
at
org.apache.spark.sql.test.TestSparkSession.sessionState(TestSQLContext.scala:41)
at
org.apache.spark.sql.SparkSession$$anonfun$1$$anonfun$apply$1.apply(SparkSession.scala:95)
at
org.apache.spark.sql.SparkSession$$anonfun$1$$anonfun$apply$1.apply(SparkSession.scala:95)
at scala.Option.map(Option.scala:146)
at
org.apache.spark.sql.SparkSession$$anonfun$1.apply(SparkSession.scala:95)
at
org.apache.spark.sql.SparkSession$$anonfun$1.apply(SparkSession.scala:94)
at org.apache.spark.sql.internal.SQLConf$.get(SQLConf.scala:126)
at
org.apache.spark.sql.catalyst.expressions.CodeGeneratorWithInterpretedFallback.createObject(CodeGeneratorWithInterpretedFallback.scala:54)
at
org.apache.spark.sql.catalyst.expressions.UnsafeProjection$.create(Projection.scala:157)
at
org.apache.spark.sql.catalyst.expressions.UnsafeProjection$.create(Projection.scala:150)
at
org.apache.spark.sql.execution.UnsafeRowSerializerSuite.org$apache$spark$sql$execution$UnsafeRowSerializerSuite$$unsafeRowConverter(UnsafeRowSerializerSuite.scala:54)
at
org.apache.spark.sql.execution.UnsafeRowSerializerSuite.org$apache$spark$sql$execution$UnsafeRowSerializerSuite$$toUnsafeRow(UnsafeRowSerializerSuite.scala:49)
at
org.apache.spark.sql.execution.UnsafeRowSerializerSuite$$anonfun$2.apply(UnsafeRowSerializerSuite.scala:63)
at
org.apache.spark.sql.execution.UnsafeRowSerializerSuite$$anonfun$2.apply(UnsafeRowSerializerSuite.scala:60)
...
```
## How was this patch tested?
N/A
Author: Wenchen Fan <wenchen@databricks.com>
Closes #21518 from cloud-fan/test.
(commit: 01452ea9c75ff027ceeb8314368c6bbedefdb2bf)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/LocalSparkSession.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/UnsafeRowSerializerSuite.scala (diff)
Commit 1d7db65e968de1c601e7f8b1ec9bc783ef2dbd01 by srowen
docs: fix typo
no => no[t]
## What changes were proposed in this pull request?
Fixing a typo.
## How was this patch tested?
Visual check of the docs.
Please review http://spark.apache.org/contributing.html before opening a
pull request.
Author: Tom Saleeba <tom.saleeba@gmail.com>
Closes #21496 from tomsaleeba/patch-1.
(commit: 1d7db65e968de1c601e7f8b1ec9bc783ef2dbd01)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/Column.scala (diff)
Commit 5d6a53d9831cc1e2115560db5cebe0eea2565dcd by meng
[SPARK-15064][ML] Locale support in StopWordsRemover
## What changes were proposed in this pull request?
Add locale support for `StopWordsRemover`.
## How was this patch tested?
[Scala|Python] unit tests.
Author: Lee Dongjin <dongjin@apache.org>
Closes #21501 from dongjinleekr/feature/SPARK-15064.
(commit: 5d6a53d9831cc1e2115560db5cebe0eea2565dcd)
The file was modifiedpython/pyspark/ml/tests.py (diff)
The file was modifiedmllib/src/main/scala/org/apache/spark/ml/feature/StopWordsRemover.scala (diff)
The file was modifiedpython/pyspark/ml/feature.py (diff)
The file was modifiedmllib/src/test/scala/org/apache/spark/ml/feature/StopWordsRemoverSuite.scala (diff)
Commit 2824f1436bb0371b7216730455f02456ef8479ce by gatorsmile
[SPARK-24531][TESTS] Remove version 2.2.0 from testing versions in
HiveExternalCatalogVersionsSuite
## What changes were proposed in this pull request?
Removing version 2.2.0 from testing versions in
HiveExternalCatalogVersionsSuite as it is not present anymore in the
mirrors and this is blocking all the open PRs.
## How was this patch tested?
running UTs
Author: Marco Gaido <marcogaido91@gmail.com>
Closes #21540 from mgaido91/SPARK-24531.
(commit: 2824f1436bb0371b7216730455f02456ef8479ce)
The file was modifiedsql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala (diff)