FailedConsole Output

Skipping 14,755 KB.. Full Log
AAAAAAP9b85aBtbiIQT2jNKU4P0+vIKc0PTOvWC8xrzgzOT8nv0gvODO3ICfVoyQ3xy+/JNU2Yj/Tagmf50wMjD4M7CWJ6SCJEgYhn6zEskT9nMS8dP3gkqLMvHTriiIGKaihyfl5xfk5qXrOEBpkDgMEMDIxMFQUlDDI2RQXJOYpFJdU5qTaKoEttlJQNjBwdnF2sVayAwD4vuoDpQAAAA==isInCollectionCheckExceptionMessage started
[info] Test test.org.apache.spark.sql.JavaColumnExpressionSuite.isInCollectionWorksCorrectlyOnJava started
[info] Test run finished: 0 failed, 0 ignored, 2 total, 0.326s
[info] Test run started
[info] Test test.org.apache.spark.sql.execution.sort.RecordBinaryComparatorSuite.testBinaryComparatorForSingleColumnRow started
[info] Test test.org.apache.spark.sql.execution.sort.RecordBinaryComparatorSuite.testBinaryComparatorForArrayColumn started
[info] Test test.org.apache.spark.sql.execution.sort.RecordBinaryComparatorSuite.testBinaryComparatorWhenOnlyTheLastColumnDiffers started
[info] Test test.org.apache.spark.sql.execution.sort.RecordBinaryComparatorSuite.testBinaryComparatorForMixedColumns started
[info] Test test.org.apache.spark.sql.execution.sort.RecordBinaryComparatorSuite.testBinaryComparatorForNullColumns started
[info] Test test.org.apache.spark.sql.execution.sort.RecordBinaryComparatorSuite.testBinaryComparatorForMultipleColumnRow started
[info] Test test.org.apache.spark.sql.execution.sort.RecordBinaryComparatorSuite.testBinaryComparatorWhenSubtractionIsDivisibleByMaxIntValue started
[info] Test test.org.apache.spark.sql.execution.sort.RecordBinaryComparatorSuite.testBinaryComparatorWhenSubtractionCanOverflowLongValue started
[info] Test run finished: 0 failed, 0 ignored, 8 total, 0.004s
[info] Test run started
[info] Test test.org.apache.spark.sql.JavaDatasetAggregatorSuite.testTypedAggregationCount started
[info] Test test.org.apache.spark.sql.JavaDatasetAggregatorSuite.testTypedAggregationSumDouble started
[info] Test test.org.apache.spark.sql.JavaDatasetAggregatorSuite.testTypedAggregationSumLong started
[info] Test test.org.apache.spark.sql.JavaDatasetAggregatorSuite.testTypedAggregationAnonClass started
[info] Test test.org.apache.spark.sql.JavaDatasetAggregatorSuite.testTypedAggregationAverage started
[info] Test run finished: 0 failed, 0 ignored, 5 total, 3.269s
[info] Test run started
[info] Test test.org.apache.spark.sql.JavaRowSuite.constructSimpleRow started
[info] Test test.org.apache.spark.sql.JavaRowSuite.constructComplexRow started
[info] Test run finished: 0 failed, 0 ignored, 2 total, 0.002s
[info] - udf/udf-group-analytics.sql - Scala UDF (53 seconds, 962 milliseconds)
[info] - udf/udf-group-analytics.sql - Regular Python UDF (7 minutes, 10 seconds)
[info] - udf/udf-group-analytics.sql - Scalar Pandas UDF (7 minutes, 14 seconds)
[info] - udf/udf-outer-join.sql - Scala UDF (24 seconds, 77 milliseconds)
[info] - udf/udf-outer-join.sql - Regular Python UDF (1 minute, 33 seconds)
[info] - udf/udf-outer-join.sql - Scalar Pandas UDF (1 minute, 46 seconds)
[info] - udf/udf-cross-join.sql - Scala UDF (2 seconds, 667 milliseconds)
[info] - udf/udf-cross-join.sql - Regular Python UDF (7 seconds, 4 milliseconds)
[info] - udf/udf-cross-join.sql - Scalar Pandas UDF (6 seconds, 400 milliseconds)
[info] - udf/udf-pivot.sql - Scala UDF (1 minute, 18 seconds)
[info] - udf/udf-pivot.sql - Regular Python UDF (5 minutes, 35 seconds)
[info] - udf/udf-pivot.sql - Scalar Pandas UDF (6 minutes, 7 seconds)
[info] - udf/udf-inner-join.sql - Scala UDF (1 second, 869 milliseconds)
[info] - udf/udf-inner-join.sql - Regular Python UDF (3 seconds, 857 milliseconds)
[info] - udf/udf-inner-join.sql - Scalar Pandas UDF (3 seconds, 167 milliseconds)
[info] - udf/udf-count.sql - Scala UDF (26 seconds, 379 milliseconds)
[info] - udf/udf-count.sql - Regular Python UDF (30 seconds, 809 milliseconds)
[info] - udf/udf-count.sql - Scalar Pandas UDF (30 seconds, 270 milliseconds)
[info] - udf/udf-having.sql - Scala UDF (7 seconds, 908 milliseconds)
[info] - udf/udf-having.sql - Regular Python UDF (50 seconds, 901 milliseconds)
[info] - udf/udf-having.sql - Scalar Pandas UDF (52 seconds, 14 milliseconds)
[info] - udf/udf-except-all.sql - Scala UDF (1 minute, 38 seconds)
[info] - udf/udf-except-all.sql - Regular Python UDF (1 minute, 57 seconds)
[info] - udf/udf-except-all.sql - Scalar Pandas UDF (2 minutes, 16 seconds)
[info] - udf/udf-intersect-all.sql - Scala UDF (1 minute, 34 seconds)
[info] - udf/udf-intersect-all.sql - Regular Python UDF (2 minutes, 37 seconds)
[info] - udf/udf-intersect-all.sql - Scalar Pandas UDF (2 minutes, 7 seconds)
[info] - udf/udf-udaf.sql - Scala UDF (1 second, 304 milliseconds)
[info] - udf/udf-udaf.sql - Regular Python UDF (1 second, 688 milliseconds)
[info] - udf/udf-udaf.sql - Scalar Pandas UDF (1 second, 651 milliseconds)
[info] - udf/udf-except.sql - Scala UDF (17 seconds, 44 milliseconds)
[info] - udf/udf-except.sql - Regular Python UDF (23 seconds, 623 milliseconds)
[info] - udf/udf-except.sql - Scalar Pandas UDF (22 seconds, 246 milliseconds)
[info] - udf/udf-inline-table.sql - Scala UDF (2 seconds, 337 milliseconds)
[info] - udf/udf-inline-table.sql - Regular Python UDF (4 seconds, 851 milliseconds)
[info] - udf/udf-inline-table.sql - Scalar Pandas UDF (3 seconds, 930 milliseconds)
[info] - udf/udf-special-values.sql - Scala UDF (1 second, 362 milliseconds)
[info] - udf/udf-special-values.sql - Regular Python UDF (2 seconds, 615 milliseconds)
[info] - udf/udf-special-values.sql - Scalar Pandas UDF (2 seconds, 991 milliseconds)
[info] - udf/udf-natural-join.sql - Scala UDF (1 minute, 8 seconds)
[info] - udf/udf-natural-join.sql - Regular Python UDF (1 minute, 18 seconds)
[info] - udf/udf-natural-join.sql - Scalar Pandas UDF (1 minute, 25 seconds)
[info] - udf/udf-join-empty-relation.sql - Scala UDF (10 seconds, 928 milliseconds)
[info] - udf/udf-join-empty-relation.sql - Regular Python UDF (13 seconds, 367 milliseconds)
[info] - udf/udf-join-empty-relation.sql - Scalar Pandas UDF (12 seconds, 795 milliseconds)
[info] - udf/pgSQL/udf-aggregates_part2.sql - Scala UDF (8 seconds, 394 milliseconds)
[info] - udf/pgSQL/udf-aggregates_part2.sql - Regular Python UDF (12 seconds, 916 milliseconds)
[info] - udf/pgSQL/udf-aggregates_part2.sql - Scalar Pandas UDF (14 seconds, 871 milliseconds)
[info] - udf/pgSQL/udf-aggregates_part1.sql - Scala UDF (20 seconds, 164 milliseconds)
[info] - udf/pgSQL/udf-aggregates_part1.sql - Regular Python UDF (1 minute, 43 seconds)
[info] - udf/pgSQL/udf-aggregates_part1.sql - Scalar Pandas UDF (1 minute, 52 seconds)
[info] - udf/pgSQL/udf-case.sql - Scala UDF (13 seconds, 216 milliseconds)
23:57:16.548 WARN org.apache.spark.sql.catalyst.optimizer.PullOutPythonUDFInJoinCondition: The join condition:(cast(udf(cast(coalesce(f#678918, cast(i#678935 as double), cast(j#678936 as double)) as string)) as double) = 2.0) of the join plan contains PythonUDF only, it will be moved out and the join plan will be turned to cross join.
23:57:17.009 WARN org.apache.spark.sql.catalyst.optimizer.PullOutPythonUDFInJoinCondition: The join condition:cast(udf(cast((coalesce(f#678918, cast(i#678935 as double)) = 2.0) as string)) as boolean) of the join plan contains PythonUDF only, it will be moved out and the join plan will be turned to cross join.
23:57:21.532 WARN org.apache.spark.sql.catalyst.optimizer.PullOutPythonUDFInJoinCondition: The join condition:(cast(udf(cast(coalesce(f#679197, cast(i#679214 as double), cast(j#679215 as double)) as string)) as double) = 2.0) of the join plan contains PythonUDF only, it will be moved out and the join plan will be turned to cross join.
23:57:21.923 WARN org.apache.spark.sql.catalyst.optimizer.PullOutPythonUDFInJoinCondition: The join condition:cast(udf(cast((coalesce(f#679197, cast(i#679214 as double)) = 2.0) as string)) as boolean) of the join plan contains PythonUDF only, it will be moved out and the join plan will be turned to cross join.
23:57:27.521 WARN org.apache.spark.sql.catalyst.optimizer.PullOutPythonUDFInJoinCondition: The join condition:(cast(udf(cast(coalesce(f#679476, cast(i#679493 as double), cast(j#679494 as double)) as string)) as double) = 2.0) of the join plan contains PythonUDF only, it will be moved out and the join plan will be turned to cross join.
23:57:27.926 WARN org.apache.spark.sql.catalyst.optimizer.PullOutPythonUDFInJoinCondition: The join condition:cast(udf(cast((coalesce(f#679476, cast(i#679493 as double)) = 2.0) as string)) as boolean) of the join plan contains PythonUDF only, it will be moved out and the join plan will be turned to cross join.
[info] - udf/pgSQL/udf-case.sql - Regular Python UDF (15 seconds, 713 milliseconds)
23:57:32.288 WARN org.apache.spark.sql.catalyst.optimizer.PullOutPythonUDFInJoinCondition: The join condition:(cast(udf(cast(coalesce(f#679755, cast(i#679772 as double), cast(j#679773 as double)) as string)) as double) = 2.0) of the join plan contains PythonUDF only, it will be moved out and the join plan will be turned to cross join.
23:57:32.849 WARN org.apache.spark.sql.catalyst.optimizer.PullOutPythonUDFInJoinCondition: The join condition:cast(udf(cast((coalesce(f#679755, cast(i#679772 as double)) = 2.0) as string)) as boolean) of the join plan contains PythonUDF only, it will be moved out and the join plan will be turned to cross join.
23:57:37.379 WARN org.apache.spark.sql.catalyst.optimizer.PullOutPythonUDFInJoinCondition: The join condition:(cast(udf(cast(coalesce(f#680034, cast(i#680051 as double), cast(j#680052 as double)) as string)) as double) = 2.0) of the join plan contains PythonUDF only, it will be moved out and the join plan will be turned to cross join.
23:57:38.482 WARN org.apache.spark.sql.catalyst.optimizer.PullOutPythonUDFInJoinCondition: The join condition:cast(udf(cast((coalesce(f#680034, cast(i#680051 as double)) = 2.0) as string)) as boolean) of the join plan contains PythonUDF only, it will be moved out and the join plan will be turned to cross join.
23:57:43.180 WARN org.apache.spark.sql.catalyst.optimizer.PullOutPythonUDFInJoinCondition: The join condition:(cast(udf(cast(coalesce(f#680313, cast(i#680330 as double), cast(j#680331 as double)) as string)) as double) = 2.0) of the join plan contains PythonUDF only, it will be moved out and the join plan will be turned to cross join.
23:57:43.636 WARN org.apache.spark.sql.catalyst.optimizer.PullOutPythonUDFInJoinCondition: The join condition:cast(udf(cast((coalesce(f#680313, cast(i#680330 as double)) = 2.0) as string)) as boolean) of the join plan contains PythonUDF only, it will be moved out and the join plan will be turned to cross join.
[info] - udf/pgSQL/udf-case.sql - Scalar Pandas UDF (15 seconds, 754 milliseconds)
[info] - ansi/interval.sql (4 seconds, 229 milliseconds)
23:57:48.322 WARN org.apache.spark.sql.SQLQueryTestSuite: 
=== Metrics of Analyzer/Optimizer Rules ===
Total number of runs: 15138077
Total time: 339.724388351 seconds

Rule                                                                                               Effective Time / Total Time                     Effective Runs / Total Runs                    

org.apache.spark.sql.catalyst.optimizer.Optimizer$OptimizeSubqueries                               35018207662 / 37857262728                       975 / 24432                                    
org.apache.spark.sql.catalyst.optimizer.ColumnPruning                                              3698299690 / 16039372484                        10512 / 114213                                 
org.apache.spark.sql.execution.datasources.FindDataSourceTable                                     12696147339 / 12951969257                       3333 / 88800                                   
org.apache.spark.sql.catalyst.expressions.codegen.package$ExpressionCanonicalizer$CleanExpressions 372100963 / 10325667061                         168972 / 6143888                               
org.apache.spark.sql.catalyst.optimizer.CollapseProject                                            2586316999 / 6240545169                         11127 / 89805                                  
org.apache.spark.sql.catalyst.optimizer.RemoveNoopOperators                                        476381398 / 5849241796                          3471 / 114189                                  
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveSubquery                                    4159269121 / 5059033826                         1107 / 88872                                   
org.apache.spark.sql.catalyst.optimizer.BooleanSimplification                                      22853753 / 5044738847                           87 / 65373                                     
org.apache.spark.sql.catalyst.optimizer.ConstantFolding                                            1435079083 / 4950268820                         7998 / 65373                                   
org.apache.spark.sql.catalyst.optimizer.LikeSimplification                                         0 / 4902342046                                  0 / 65373                                      
org.apache.spark.sql.catalyst.optimizer.SimplifyCasts                                              315659043 / 4872177667                          2226 / 65373                                   
org.apache.spark.sql.catalyst.optimizer.RemoveDispensableExpressions                               0 / 4871759781                                  0 / 65373                                      
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveReferences                                  2444544682 / 4723297587                         12822 / 89091                                  
org.apache.spark.sql.catalyst.optimizer.CombineUnions                                              17741801 / 4675037550                           45 / 89829                                     
org.apache.spark.sql.catalyst.optimizer.PushDownPredicates                                         1000213078 / 4654918019                         3261 / 65397                                   
org.apache.spark.sql.catalyst.optimizer.NullPropagation                                            96542435 / 4515061708                           741 / 65397                                    
org.apache.spark.sql.catalyst.optimizer.PruneFilters                                               137424251 / 4472612412                          822 / 65373                                    
org.apache.spark.sql.catalyst.optimizer.TransposeWindow                                            0 / 4259391597                                  0 / 65397                                      
org.apache.spark.sql.catalyst.optimizer.ReorderAssociativeOperator                                 0 / 4218615721                                  0 / 65373                                      
org.apache.spark.sql.catalyst.optimizer.SimplifyBinaryComparison                                   8614099 / 4123394740                            33 / 65373                                     
org.apache.spark.sql.catalyst.optimizer.SimplifyCaseConversionExpressions                          0 / 3964681179                                  0 / 65373                                      
org.apache.spark.sql.catalyst.optimizer.RewriteCorrelatedScalarSubquery                            30774839 / 3949134757                           63 / 65373                                     
org.apache.spark.sql.catalyst.optimizer.OptimizeIn                                                 1875627 / 3907008497                            12 / 65397                                     
org.apache.spark.sql.catalyst.optimizer.SimplifyExtractValueOps                                    1480614 / 3900265362                            9 / 65373                                      
org.apache.spark.sql.catalyst.optimizer.SimplifyConditionals                                       38537291 / 3876821292                           549 / 65373                                    
org.apache.spark.sql.catalyst.optimizer.EliminateResolvedHint                                      0 / 3861454378                                  0 / 24432                                      
org.apache.spark.sql.catalyst.optimizer.ReplaceNullWithFalseInPredicate                            799198 / 3804164614                             9 / 65373                                      
org.apache.spark.sql.catalyst.optimizer.PushDownLeftSemiAntiJoin                                   290139086 / 3764645383                          639 / 65397                                    
org.apache.spark.sql.catalyst.optimizer.EliminateOuterJoin                                         141978457 / 3674455968                          195 / 65397                                    
org.apache.spark.sql.catalyst.optimizer.ReorderJoin                                                21195025 / 3672234402                           24 / 65397                                     
org.apache.spark.sql.catalyst.optimizer.PushLeftSemiLeftAntiThroughJoin                            3337269 / 3642874644                            9 / 65397                                      
org.apache.spark.sql.catalyst.optimizer.CollapseRepartition                                        0 / 3631498236                                  0 / 65397                                      
org.apache.spark.sql.catalyst.optimizer.LimitPushDown                                              0 / 3505267251                                  0 / 65397                                      
org.apache.spark.sql.catalyst.optimizer.PushProjectionThroughUnion                                 17643280 / 3468009070                           33 / 65397                                     
org.apache.spark.sql.catalyst.optimizer.CombineLimits                                              0 / 3408527686                                  0 / 65397                                      
org.apache.spark.sql.catalyst.optimizer.ConstantPropagation                                        2897376 / 3370191138                            12 / 65397                                     
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveAggregateFunctions                          1349064020 / 3347123585                         453 / 88863                                    
org.apache.spark.sql.catalyst.optimizer.CombineFilters                                             0 / 3334090878                                  0 / 65397                                      
org.apache.spark.sql.catalyst.optimizer.CollapseWindow                                             0 / 3279315750                                  0 / 65397                                      
org.apache.spark.sql.catalyst.optimizer.EliminateSerialization                                     0 / 3247603545                                  0 / 65373                                      
org.apache.spark.sql.catalyst.optimizer.EliminateSorts                                             1831968 / 3194998041                            18 / 65373                                     
org.apache.spark.sql.catalyst.optimizer.InferFiltersFromConstraints                                821656689 / 2814937256                          1482 / 24408                                   
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveDeserializer                                1809639284 / 2738922896                         1881 / 89091                                   
org.apache.spark.sql.catalyst.optimizer.FoldablePropagation                                        199906208 / 2713685346                          393 / 65397                                    
org.apache.spark.sql.catalyst.optimizer.PropagateEmptyRelation                                     111787294 / 2653697332                          771 / 49611                                    
org.apache.spark.sql.catalyst.analysis.EliminateSubqueryAliases                                    1864523530 / 2237639602                         12651 / 24432                                  
org.apache.spark.sql.catalyst.optimizer.PullupCorrelatedPredicates                                 546645496 / 2202876933                          606 / 24432                                    
org.apache.spark.sql.catalyst.optimizer.ComputeCurrentTime                                         3068866 / 2199257460                            9 / 24432                                      
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveFunctions                                   1144918708 / 2044292407                         10737 / 88872                                  
org.apache.spark.sql.catalyst.optimizer.ReplaceExpressions                                         21655035 / 2041819027                           159 / 24432                                    
org.apache.spark.sql.catalyst.optimizer.GetCurrentDatabase                                         0 / 1928291000                                  0 / 24432                                      
org.apache.spark.sql.catalyst.analysis.DecimalPrecision                                            384330108 / 1905527524                          3660 / 88800                                   
org.apache.spark.sql.catalyst.optimizer.RemoveRedundantAliases                                     44777741 / 1890504806                           315 / 65373                                    
org.apache.spark.sql.execution.python.ExtractPythonUDFs                                            713160668 / 1876565847                          1272 / 24408                                   
org.apache.spark.sql.catalyst.analysis.TypeCoercion$ImplicitTypeCasts                              398570304 / 1875109248                          1965 / 88800                                   
org.apache.spark.sql.catalyst.optimizer.ReplaceExceptWithFilter                                    47715956 / 1831323640                           54 / 25311                                     
org.apache.spark.sql.catalyst.analysis.ResolveTimeZone                                             1282483730 / 1803836876                         24531 / 88800                                  
org.apache.spark.sql.catalyst.optimizer.ReplaceIntersectWithSemiJoin                               28334953 / 1787932386                           87 / 25311                                     
org.apache.spark.sql.execution.datasources.PruneFileSourcePartitions                               121210033 / 1749410704                          6 / 24408                                      
org.apache.spark.sql.catalyst.optimizer.ReassignLambdaVariableID                                   0 / 1734650534                                  0 / 24408                                      
org.apache.spark.sql.catalyst.optimizer.ReplaceDistinctWithAggregate                               151821972 / 1726016050                          639 / 25311                                    
org.apache.spark.sql.catalyst.optimizer.RewriteIntersectAll                                        97154097 / 1723645554                           156 / 25311                                    
org.apache.spark.sql.catalyst.optimizer.RewriteExceptAll                                           93055634 / 1658760244                           204 / 25311                                    
org.apache.spark.sql.catalyst.analysis.Analyzer$ExtractGenerator                                   4591349 / 1625736221                            24 / 88950                                     
org.apache.spark.sql.catalyst.optimizer.EliminateMapObjects                                        0 / 1617030160                                  0 / 24408                                      
org.apache.spark.sql.catalyst.optimizer.ReplaceExceptWithAntiJoin                                  74218395 / 1615931146                           180 / 25311                                    
org.apache.spark.sql.catalyst.optimizer.RemoveRepetitionFromGroupExpressions                       11257106 / 1509349662                           93 / 24549                                     
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveAliases                                     830793410 / 1483095938                          11469 / 88872                                  
org.apache.spark.sql.catalyst.optimizer.RewriteDistinctAggregates                                  30037309 / 1478426708                           15 / 24432                                     
org.apache.spark.sql.catalyst.analysis.EliminateView                                               0 / 1452738624                                  0 / 24432                                      
org.apache.spark.sql.catalyst.optimizer.RemoveLiteralFromGroupExpressions                          2307073 / 1447416115                            24 / 24549                                     
org.apache.spark.sql.catalyst.optimizer.RewritePredicateSubquery                                   365674198 / 1401991583                          912 / 24408                                    
org.apache.spark.sql.catalyst.optimizer.OptimizeLimitZero                                          0 / 1401244248                                  0 / 24432                                      
org.apache.spark.sql.catalyst.analysis.UpdateAttributeNullability                                  90722345 / 1396679728                           567 / 43140                                    
org.apache.spark.sql.catalyst.optimizer.ReplaceDeduplicateWithAggregate                            0 / 1379424890                                  0 / 24432                                      
org.apache.spark.sql.catalyst.analysis.TypeCoercion$PromoteStrings                                 236293317 / 1355625526                          2244 / 88800                                   
org.apache.spark.sql.catalyst.optimizer.DecimalAggregates                                          0 / 1307196533                                  0 / 24408                                      
org.apache.spark.sql.catalyst.analysis.TypeCoercion$FunctionArgumentConversion                     259207489 / 1298577599                          1011 / 88800                                   
org.apache.spark.sql.catalyst.analysis.TypeCoercion$InConversion                                   69517641 / 1291500897                           399 / 88800                                    
org.apache.spark.sql.execution.python.ExtractPythonUDFFromAggregate                                116962769 / 1289657885                          486 / 24408                                    
org.apache.spark.sql.catalyst.optimizer.PushPredicateThroughNonJoin                                18545725 / 1275092128                           30 / 24408                                     
org.apache.spark.sql.execution.python.ExtractGroupingPythonUDFFromAggregate                        9797327 / 1255352925                            18 / 24408                                     
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations                                   1009357641 / 1211463879                         17835 / 89229                                  
org.apache.spark.sql.catalyst.optimizer.NormalizeFloatingNumbers                                   23309630 / 1149019399                           18 / 24408                                     
org.apache.spark.sql.catalyst.optimizer.CombineTypedFilters                                        0 / 1117689775                                  0 / 24408                                      
org.apache.spark.sql.catalyst.optimizer.RemoveRedundantSorts                                       0 / 1115416771                                  0 / 24408                                      
org.apache.spark.sql.catalyst.optimizer.PullOutPythonUDFInJoinCondition                            7255769 / 1079572505                            12 / 24408                                     
org.apache.spark.sql.catalyst.optimizer.ObjectSerializerPruning                                    0 / 1079453304                                  0 / 24408                                      
org.apache.spark.sql.catalyst.optimizer.CheckCartesianProducts                                     0 / 1048697772                                  0 / 24408                                      
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRandomSeed                                  420525 / 1019198941                             3 / 88800                                      
org.apache.spark.sql.catalyst.analysis.TypeCoercion$DateTimeOperations                             9735177 / 1005143067                            84 / 88800                                     
org.apache.spark.sql.catalyst.analysis.TypeCoercion$BooleanEquality                                7901109 / 994941710                             204 / 88800                                    
org.apache.spark.sql.catalyst.analysis.TypeCoercion$IfCoercion                                     30781728 / 974921070                            195 / 88800                                    
org.apache.spark.sql.catalyst.analysis.TimeWindowing                                               0 / 966516191                                   0 / 88863                                      
org.apache.spark.sql.catalyst.analysis.ResolveCreateNamedStruct                                    19545671 / 931275443                            174 / 89091                                    
org.apache.spark.sql.catalyst.analysis.Analyzer$LookupFunctions                                    0 / 924863532                                   0 / 43584                                      
org.apache.spark.sql.catalyst.analysis.TypeCoercion$Division                                       39796000 / 919770287                            297 / 88800                                    
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveMissingReferences                           41674275 / 871982590                            120 / 88950                                    
org.apache.spark.sql.catalyst.analysis.TypeCoercion$ConcatCoercion                                 10010415 / 866193874                            48 / 88800                                     
org.apache.spark.sql.catalyst.analysis.TypeCoercion$EltCoercion                                    3259578 / 858699118                             9 / 88800                                      
org.apache.spark.sql.catalyst.analysis.ResolveHigherOrderFunctions                                 15544620 / 813008702                            120 / 88800                                    
org.apache.spark.sql.catalyst.analysis.ResolveInlineTables                                         470669339 / 757898741                           3186 / 88803                                   
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveWindowOrder                                 3477186 / 755994854                             3 / 88863                                      
org.apache.spark.sql.catalyst.analysis.TypeCoercion$CaseWhenCoercion                               29186090 / 727471861                            186 / 88800                                    
org.apache.spark.sql.catalyst.analysis.Analyzer$GlobalAggregates                                   23294450 / 722293796                            1065 / 88863                                   
org.apache.spark.sql.catalyst.analysis.ResolveLambdaVariables                                      33357763 / 711285742                            117 / 88800                                    
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveWindowFrame                                 8830870 / 706911556                             69 / 88863                                     
org.apache.spark.sql.catalyst.analysis.CleanupAliases                                              293761248 / 706194574                           14265 / 57405                                  
org.apache.spark.sql.catalyst.analysis.TypeCoercion$MapZipWithCoercion                             12385986 / 695293360                            39 / 88800                                     
org.apache.spark.sql.catalyst.analysis.TypeCoercion$WindowFrameCoercion                            3885499 / 682556021                             30 / 88800                                     
org.apache.spark.sql.catalyst.analysis.Analyzer$ExtractWindowExpressions                           69912942 / 671253804                            129 / 88863                                    
org.apache.spark.sql.catalyst.analysis.TypeCoercion$StackCoercion                                  0 / 645445435                                   0 / 88800                                      
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveUpCast                                      75059690 / 590423531                            1248 / 89091                                   
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveNewInstance                                 0 / 577299298                                   0 / 89091                                      
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveGroupingAnalytics                           138360156 / 561918025                           348 / 89043                                    
org.apache.spark.sql.catalyst.analysis.ResolveHints$ResolveJoinStrategyHints                       0 / 543337524                                   0 / 43599                                      
org.apache.spark.sql.catalyst.analysis.Analyzer$PullOutNondeterministic                            4179182 / 503614043                             3 / 43140                                      
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolvePivot                                       178464613 / 459251299                           234 / 88971                                    
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveTables                                      0 / 424318210                                   0 / 89229                                      
org.apache.spark.sql.catalyst.analysis.Analyzer$HandleNullInputsForUDF                             3307112 / 422718378                             6 / 43140                                      
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveOrdinalInOrderByAndGroupBy                  3509348 / 414441352                             96 / 88950                                     
org.apache.spark.sql.execution.datasources.DataSourceResolution                                    82046847 / 378057737                            603 / 88800                                    
org.apache.spark.sql.catalyst.analysis.TypeCoercion$WidenSetOperationTypes                         46082427 / 363792267                            249 / 88800                                    
org.apache.spark.sql.execution.datasources.PreprocessTableInsertion                                178142645 / 327809913                           1890 / 43140                                   
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveAggAliasInGroupBy                           8485432 / 321450851                             51 / 88950                                     
org.apache.spark.sql.catalyst.analysis.CTESubstitution                                             53567494 / 303846642                            204 / 43914                                    
org.apache.spark.sql.catalyst.analysis.ResolveTableValuedFunctions                                 43697524 / 296037911                            207 / 89229                                    
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveAlterTable                                  0 / 286330988                                   0 / 89229                                      
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveNaturalAndUsingJoin                         38023890 / 282085383                            171 / 88863                                    
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveSubqueryColumnAliases                       12121432 / 278296001                            246 / 88866                                    
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveGenerate                                    2542262 / 272089806                             12 / 88950                                     
org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveOutputRelation                              0 / 271942619                                   0 / 88863                                      
org.apache.spark.sql.execution.datasources.ResolveSQLOnFile                                        0 / 258855371                                   0 / 88800                                      
org.apache.spark.sql.catalyst.optimizer.CombineConcats                                             2638354 / 256732287                             21 / 65373                                     
org.apache.spark.sql.catalyst.analysis.ResolveHints$ResolveCoalesceHints                           1444214 / 231121220                             3 / 43599                                      
org.apache.spark.sql.execution.datasources.DataSourceAnalysis                                      100672574 / 230486342                           2127 / 43140                                   
org.apache.spark.sql.execution.datasources.FallBackFileSourceV2                                    0 / 223899859                                   0 / 88800                                      
org.apache.spark.sql.execution.datasources.PreprocessTableCreation                                 520228 / 221482808                              3 / 43143                                      
org.apache.spark.sql.catalyst.optimizer.CostBasedJoinReorder                                       0 / 181981183                                   0 / 24408                                      
org.apache.spark.sql.catalyst.analysis.UpdateOuterReferences                                       9276517 / 174167959                             27 / 43140                                     
org.apache.spark.sql.catalyst.optimizer.EliminateDistinct                                          0 / 164598293                                   0 / 24432                                      
org.apache.spark.sql.catalyst.analysis.SubstituteUnresolvedOrdinals                                10154197 / 163254520                            123 / 43914                                    
org.apache.spark.sql.catalyst.analysis.Analyzer$WindowsSubstitution                                3538151 / 132516916                             9 / 43914                                      
org.apache.spark.sql.catalyst.analysis.EliminateUnions                                             0 / 124759199                                   0 / 43914                                      
org.apache.spark.sql.catalyst.analysis.ResolveHints$RemoveAllHints                                 0 / 113189251                                   0 / 43599                                      
org.apache.spark.sql.execution.datasources.SchemaPruning                                           0 / 106627791                                   0 / 24408                                      
org.apache.spark.sql.execution.OptimizeMetadataOnlyQuery                                           0 / 99380240                                    0 / 24408                                      
     
23:57:53.247 WARN org.apache.spark.sql.SQLQueryTestSuite: 

===== POSSIBLE THREAD LEAK IN SUITE o.a.s.sql.SQLQueryTestSuite, thread names: Idle Worker Monitor for python3.6 =====

[info] ScalaTest
[info] Run completed in 1 hour, 47 minutes, 33 seconds.
[info] Total number of tests run: 5897
[info] Suites: completed 288, aborted 0
[info] Tests: succeeded 5897, failed 0, canceled 0, ignored 24, pending 0
[info] All tests passed.
[info] Passed: Total 6003, Failed 0, Errors 0, Passed 6003, Ignored 24
[success] Total time: 6469 s, completed Jul 21, 2019 11:57:55 PM
[warn] Multiple main classes detected.  Run 'show discoveredMainClasses' to see the list
[info] Packaging /home/jenkins/workspace/SparkPullRequestBuilder@2/core/target/scala-2.12/spark-core_2.12-3.0.0-SNAPSHOT.jar ...
[info] Done packaging.
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/parquet/ParquetWriteBuilder.scala:91: value ENABLE_JOB_SUMMARY in class ParquetOutputFormat is deprecated: see corresponding Javadoc for more information.
[warn]       && conf.get(ParquetOutputFormat.ENABLE_JOB_SUMMARY) == null) {
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala:128: value ENABLE_JOB_SUMMARY in class ParquetOutputFormat is deprecated: see corresponding Javadoc for more information.
[warn]       && conf.get(ParquetOutputFormat.ENABLE_JOB_SUMMARY) == null) {
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala:262: class ParquetInputSplit in package hadoop is deprecated: see corresponding Javadoc for more information.
[warn]         new org.apache.parquet.hadoop.ParquetInputSplit(
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala:273: method readFooter in class ParquetFileReader is deprecated: see corresponding Javadoc for more information.
[warn]         ParquetFileReader.readFooter(sharedConf, filePath, SKIP_ROW_GROUPS).getFileMetaData
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala:447: method readFooter in class ParquetFileReader is deprecated: see corresponding Javadoc for more information.
[warn]           ParquetFileReader.readFooter(
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/parquet/ParquetPartitionReaderFactory.scala:121: class ParquetInputSplit in package hadoop is deprecated: see corresponding Javadoc for more information.
[warn]           Option[TimeZone]) => RecordReader[Void, T]): RecordReader[Void, T] = {
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/parquet/ParquetPartitionReaderFactory.scala:126: class ParquetInputSplit in package hadoop is deprecated: see corresponding Javadoc for more information.
[warn]       new org.apache.parquet.hadoop.ParquetInputSplit(
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/parquet/ParquetPartitionReaderFactory.scala:135: method readFooter in class ParquetFileReader is deprecated: see corresponding Javadoc for more information.
[warn]       ParquetFileReader.readFooter(conf, filePath, SKIP_ROW_GROUPS).getFileMetaData
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/parquet/ParquetPartitionReaderFactory.scala:184: class ParquetInputSplit in package hadoop is deprecated: see corresponding Javadoc for more information.
[warn]       split: ParquetInputSplit,
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/parquet/ParquetPartitionReaderFactory.scala:213: class ParquetInputSplit in package hadoop is deprecated: see corresponding Javadoc for more information.
[warn]       split: ParquetInputSplit,
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala:283: class WriteToDataSourceV2 in package v2 is deprecated (since 2.4.0): Use specific logical plans like AppendData instead
[warn]               WriteToDataSourceV2(write, df.logicalPlan)
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/WriteToMicroBatchDataSource.scala:36: class WriteToDataSourceV2 in package v2 is deprecated (since 2.4.0): Use specific logical plans like AppendData instead
[warn]   def createPlan(batchId: Long): WriteToDataSourceV2 = {
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/WriteToMicroBatchDataSource.scala:37: class WriteToDataSourceV2 in package v2 is deprecated (since 2.4.0): Use specific logical plans like AppendData instead
[warn]     WriteToDataSourceV2(new MicroBatchWrite(batchId, write), query)
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala:474: method poll in class KafkaConsumer is deprecated: see corresponding Javadoc for more information.
[warn]     val p = consumer.poll(pollTimeoutMs)
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReader.scala:119: method poll in trait Consumer is deprecated: see corresponding Javadoc for more information.
[warn]     consumer.poll(0)
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReader.scala:167: method poll in trait Consumer is deprecated: see corresponding Javadoc for more information.
[warn]         consumer.poll(0)
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReader.scala:215: method poll in trait Consumer is deprecated: see corresponding Javadoc for more information.
[warn]       consumer.poll(0)
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReader.scala:245: method poll in trait Consumer is deprecated: see corresponding Javadoc for more information.
[warn]       consumer.poll(0)
[warn] 
[warn] /home/jenkins/workspace/SparkPullRequestBuilder@2/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReader.scala:321: method poll in trait Consumer is deprecated: see corresponding Javadoc for more information.
[warn]           consumer.poll(0)
[warn] 
[info] KafkaDontFailOnDataLossSuite:
[info] - failOnDataLoss=false should not return duplicated records: microbatch v1 (33 seconds, 426 milliseconds)
[info] - failOnDataLoss=false should not return duplicated records: microbatch v2 (8 seconds, 657 milliseconds)
[info] - failOnDataLoss=false should not return duplicated records: continuous processing (7 seconds, 704 milliseconds)
[info] - failOnDataLoss=false should not return duplicated records: batch v1 (8 seconds, 406 milliseconds)
[info] - failOnDataLoss=false should not return duplicated records: batch v2 (7 seconds, 723 milliseconds)
[info] KafkaSourceStressForDontFailOnDataLossSuite:
[info] - stress test for failOnDataLoss=false (20 seconds, 981 milliseconds)
[info] KafkaRelationSuiteV1:
[info] - explicit earliest to latest offsets (3 seconds, 894 milliseconds)
[info] - default starting and ending offsets (2 seconds, 591 milliseconds)
[info] - explicit offsets (4 seconds, 199 milliseconds)
[info] - reuse same dataframe in query (2 seconds, 132 milliseconds)
[info] - test late binding start offsets (5 seconds, 678 milliseconds)
[info] - bad batch query options (25 milliseconds)
[info] - allow group.id prefix (4 seconds, 852 milliseconds)
[info] - allow group.id override (2 seconds, 939 milliseconds)
[info] - read Kafka transactional messages: read_committed (10 seconds, 317 milliseconds)
[info] - read Kafka transactional messages: read_uncommitted (5 seconds, 887 milliseconds)
[info] - V1 Source is used when set through SQLConf (10 milliseconds)
[info] KafkaRelationSuiteV2:
[info] - explicit earliest to latest offsets (3 seconds, 941 milliseconds)
[info] - default starting and ending offsets (3 seconds, 626 milliseconds)
[info] - explicit offsets (4 seconds, 715 milliseconds)
[info] - reuse same dataframe in query (1 second, 892 milliseconds)
[info] - test late binding start offsets (5 seconds, 577 milliseconds)
[info] - bad batch query options (186 milliseconds)
[info] - allow group.id prefix (2 seconds, 981 milliseconds)
[info] - allow group.id override (2 seconds, 493 milliseconds)
[info] - read Kafka transactional messages: read_committed (8 seconds, 334 milliseconds)
[info] - read Kafka transactional messages: read_uncommitted (6 seconds, 527 milliseconds)
[info] - V2 Source is used when set through SQLConf (8 milliseconds)
[info] KafkaDataConsumerSuite:
[info] - SPARK-19886: Report error cause correctly in reportDataLoss (10 milliseconds)
[info] - SPARK-23623: concurrent use of KafkaDataConsumer (1 minute, 12 seconds)
[info] KafkaContinuousSourceSuite:
[info] - cannot stop Kafka stream (1 second, 614 milliseconds)
[info] - assign from latest offsets (failOnDataLoss: true) (8 seconds, 946 milliseconds)
[info] - assign from earliest offsets (failOnDataLoss: true) (7 seconds, 22 milliseconds)
[info] - assign from specific offsets (failOnDataLoss: true) (3 seconds, 885 milliseconds)
[info] - subscribing topic by name from latest offsets (failOnDataLoss: true) (13 seconds, 121 milliseconds)
[info] - subscribing topic by name from earliest offsets (failOnDataLoss: true) (9 seconds, 67 milliseconds)
[info] - subscribing topic by name from specific offsets (failOnDataLoss: true) (6 seconds, 112 milliseconds)
[info] - subscribing topic by pattern from latest offsets (failOnDataLoss: true) (10 seconds, 845 milliseconds)
[info] - subscribing topic by pattern from earliest offsets (failOnDataLoss: true) (12 seconds, 289 milliseconds)
[info] - subscribing topic by pattern from specific offsets (failOnDataLoss: true) (4 seconds, 796 milliseconds)
[info] - assign from latest offsets (failOnDataLoss: false) (11 seconds, 878 milliseconds)
[info] - assign from earliest offsets (failOnDataLoss: false) (6 seconds, 942 milliseconds)
[info] - assign from specific offsets (failOnDataLoss: false) (3 seconds, 994 milliseconds)
[info] - subscribing topic by name from latest offsets (failOnDataLoss: false) (11 seconds, 66 milliseconds)
[info] - subscribing topic by name from earliest offsets (failOnDataLoss: false) (11 seconds, 544 milliseconds)
[info] - subscribing topic by name from specific offsets (failOnDataLoss: false) (4 seconds, 550 milliseconds)
Attempting to post to Github...
[error] running /home/jenkins/workspace/SparkPullRequestBuilder@2/build/sbt -Phadoop-2.7 -Phive-thriftserver -Phive -Dtest.exclude.tags=org.apache.spark.tags.ExtendedHiveTest,org.apache.spark.tags.ExtendedYarnTest hive-thriftserver/test avro/test mllib/test hive/test repl/test catalyst/test sql/test sql-kafka-0-10/test examples/test ; process was terminated by signal 9
 > Post successful.
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/107980/
Test FAILed.
Finished: FAILURE