SuccessChanges

Summary

  1. [SPARK-23525][SQL] Support ALTER TABLE CHANGE COLUMN COMMENT for (commit: 8cd6a96535bd8b8c4e40b87869ba02f916a997ed) (details)
  2. [SPARK-23020][CORE][BRANCH-2.3] Fix another race in the in-process (commit: ee6e79737cab6078dc94ddc8fd02433e8199104f) (details)
Commit 8cd6a96535bd8b8c4e40b87869ba02f916a997ed by wenchen
[SPARK-23525][SQL] Support ALTER TABLE CHANGE COLUMN COMMENT for
external hive table
## What changes were proposed in this pull request?
The following query doesn't work as expected:
``` CREATE EXTERNAL TABLE ext_table(a STRING, b INT, c STRING)
PARTITIONED BY (d STRING) LOCATION 'sql/core/spark-warehouse/ext_table';
ALTER TABLE ext_table CHANGE a a STRING COMMENT "new comment"; DESC
ext_table;
``` The comment of column `a` is not updated, that's because
`HiveExternalCatalog.doAlterTable` ignores table schema changes. To fix
the issue, we should call `doAlterTableDataSchema` instead of
`doAlterTable`.
## How was this patch tested?
Updated `DDLSuite.testChangeColumn`.
Author: Xingbo Jiang <xingbo.jiang@databricks.com>
Closes #20696 from jiangxb1987/alterColumnComment.
(cherry picked from commit ac76eff6a88f6358a321b84cb5e60fb9d6403419)
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
(commit: 8cd6a96535bd8b8c4e40b87869ba02f916a997ed)
The file was modifiedsql/core/src/test/resources/sql-tests/results/change-column.sql.out (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala (diff)
The file was modifiedsql/core/src/test/resources/sql-tests/inputs/change-column.sql (diff)
Commit ee6e79737cab6078dc94ddc8fd02433e8199104f by wenchen
[SPARK-23020][CORE][BRANCH-2.3] Fix another race in the in-process
launcher test.
First the bad news: there's an unfixable race in the launcher code.
(By unfixable I mean it would take a lot more effort than this change to
fix it.) The good news is that it should only affect super short lived
applications, such as the one run by the flaky test, so it's possible to
work around it in our test.
The fix also uncovered an issue with the recently added "closeAndWait()"
method; closing the connection would still possibly cause data loss, so
this change waits a while for the connection to finish itself, and
closes the socket if that times out. The existing connection timeout is
reused so that if desired it's possible to control how long to wait.
As part of that I also restored the old behavior that disconnect() would
force a disconnection from the child app; the "wait for data to arrive"
approach is only taken when disposing of the handle.
I tested this by inserting a bunch of sleeps in the test and the socket
handling code in the launcher library; with those I was able to
reproduce the error from the jenkins jobs. With the changes, even with
all the sleeps still in place, all tests pass.
Author: Marcelo Vanzin <vanzin@cloudera.com>
Closes #20743 from vanzin/SPARK-23020.
(commit: ee6e79737cab6078dc94ddc8fd02433e8199104f)
The file was modifiedlauncher/src/test/java/org/apache/spark/launcher/LauncherServerSuite.java (diff)
The file was modifiedlauncher/src/main/java/org/apache/spark/launcher/ChildProcAppHandle.java (diff)
The file was modifiedlauncher/src/main/java/org/apache/spark/launcher/AbstractAppHandle.java (diff)
The file was modifiedcore/src/test/java/org/apache/spark/launcher/SparkLauncherSuite.java (diff)
The file was modifiedlauncher/src/main/java/org/apache/spark/launcher/InProcessAppHandle.java (diff)
The file was modifiedlauncher/src/main/java/org/apache/spark/launcher/LauncherServer.java (diff)