FailedChanges

Summary

  1. [SPARK-25579][SQL] Use quoted attribute names if needed in pushed ORC (details)
Commit 2c664edc060a41340eb374fd44b5d32c3c06a15c by hyukjinkwon
[SPARK-25579][SQL] Use quoted attribute names if needed in pushed ORC
predicates
## What changes were proposed in this pull request?
This PR aims to fix an ORC performance regression at Spark 2.4.0 RCs
from Spark 2.3.2. Currently, for column names with `.`, the pushed
predicates are ignored.
**Test Data**
```scala scala> val df =
spark.range(Int.MaxValue).sample(0.2).toDF("col.with.dot") scala>
df.write.mode("overwrite").orc("/tmp/orc")
```
**Spark 2.3.2**
```scala scala> spark.sql("set spark.sql.orc.impl=native") scala>
spark.sql("set spark.sql.orc.filterPushdown=true") scala>
spark.time(spark.read.orc("/tmp/orc").where("`col.with.dot` < 10").show)
+------------+
|col.with.dot|
+------------+
|           5|
|           7|
|           8|
+------------+
Time taken: 1542 ms
scala> spark.time(spark.read.orc("/tmp/orc").where("`col.with.dot` <
10").show)
+------------+
|col.with.dot|
+------------+
|           5|
|           7|
|           8|
+------------+
Time taken: 152 ms
```
**Spark 2.4.0 RC3**
```scala scala>
spark.time(spark.read.orc("/tmp/orc").where("`col.with.dot` < 10").show)
+------------+
|col.with.dot|
+------------+
|           5|
|           7|
|           8|
+------------+
Time taken: 4074 ms
scala> spark.time(spark.read.orc("/tmp/orc").where("`col.with.dot` <
10").show)
+------------+
|col.with.dot|
+------------+
|           5|
|           7|
|           8|
+------------+
Time taken: 1771 ms
```
## How was this patch tested?
Pass the Jenkins with a newly added test case.
Closes #22597 from dongjoon-hyun/SPARK-25579.
Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by:
hyukjinkwon <gurwls223@apache.org>
The file was modifiedsql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFilters.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala (diff)
The file was modifiedsql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcTest.scala (diff)