1. [SPARK-30480][PYSPARK][TESTS] Fix 'test_memory_limit' on pyspark test (details)
  2. [SPARK-30018][SQL] Support ALTER DATABASE SET OWNER syntax (details)
Commit afd70a0f6fc1b44164e41a57dfc4fd8a5df642e1 by gurwls223
[SPARK-30480][PYSPARK][TESTS] Fix 'test_memory_limit' on pyspark test
### What changes were proposed in this pull request?
This patch increases the memory limit in the test 'test_memory_limit'
from 1m to 8m. Credit to srowen and HyukjinKwon to provide the idea of
suspicion and guide how to fix.
### Why are the changes needed?
We observed consistent Pyspark test failures on multiple PRs (#26955,
#26201, #27064) which block the PR builds whenever the test is included.
### Does this PR introduce any user-facing change?
### How was this patch tested?
Jenkins builds passed in WIP PR (#27159)
Closes #27162 from HeartSaVioR/SPARK-30480.
Authored-by: Jungtaek Lim (HeartSaVioR) <>
Signed-off-by: HyukjinKwon <>
The file was modifiedpython/pyspark/tests/ (diff)
Commit bcf07cbf5f760f0959b8178ef807cb61adec8cc3 by wenchen
### What changes were proposed in this pull request? In this pull
request, we are going to support `SET OWNER` syntax for databases and
[USER|ROLE|GROUP] user_or_role_group;
``` Before this commit,
we didn't care much about ownerships for the catalog objects. In,
we determined to use properties to store ownership staff, and
temporarily used `alter database ... set dbproperties ...` to support
switch ownership of a database. This PR aims to use the formal syntax to
replace it.
In hive, `ownerName/Type` are fields of the database objects, also they
can be normal properties.
``` create schema test1 with dbproperties('ownerName'='yaooqinn')
``` The create/alter database syntax will not change the owner to
`yaooqinn` but store it in parameters. e.g.
| db_name  | comment  |                           location             
             | owner_name  | owner_type  |      parameters       |
| test1    |          |
hdfs://quickstart.cloudera:8020/user/hive/warehouse/test1.db  |
anonymous   | USER        | {ownerName=yaooqinn}  |
``` In this pull request, because we let the `ownerName` become
reversed, so it will neither change the owner nor store in dbproperties,
just be omitted silently.
## Why are the changes needed?
Formal syntax support for changing database ownership
### Does this PR introduce any user-facing change?
yes, add a new syntax
### How was this patch tested?
add unit tests
Closes #26775 from yaooqinn/SPARK-30018.
Authored-by: Kent Yao <> Signed-off-by: Wenchen Fan
The file was modifieddocs/ (diff)
The file was modifiedsql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/CreateNamespaceExec.scala (diff)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala (diff)
The file was modifiedsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DescribeNamespaceExec.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/datasources/v2/V2SessionCatalogSuite.scala (diff)
The file was modifiedsql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/DDLParserSuite.scala (diff)
The file was modifiedsql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/ (diff)
The file was modifiedsql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala (diff)
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2SessionCatalog.scala (diff)