SuccessConsole Output

Skipping 2,564 KB.. Full Log
MEMDIxMFQUlDCw2+gXFyTm2QEAI9P8iI4AAAA=] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:330: Could not find any member to link for "ClassCastException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:323: Could not find any member to link for "ClassCastException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:315: Could not find any member to link for "ClassCastException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:308: Could not find any member to link for "ClassCastException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:300: Could not find any member to link for "ClassCastException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:293: Could not find any member to link for "ClassCastException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:286: Could not find any member to link for "ClassCastException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:279: Could not find any member to link for "ClassCastException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:272: Could not find any member to link for "ClassCastException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:265: Could not find any member to link for "ClassCastException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:258: Could not find any member to link for "ClassCastException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:251: Could not find any member to link for "ClassCastException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:243: Could not find any member to link for "NullPointerException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:234: Could not find any member to link for "NullPointerException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:226: Could not find any member to link for "NullPointerException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:218: Could not find any member to link for "NullPointerException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:202: Could not find any member to link for "NullPointerException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:210: Could not find any member to link for "NullPointerException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala:194: Could not find any member to link for "NullPointerException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala:61: Could not find any member to link for "NullPointerException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/core/src/main/scala/org/apache/spark/sql/RuntimeConfig.scala:64: Could not find any member to link for "java.util.NoSuchElementException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala:142: Could not find any member to link for "org.apache.spark.sql.internal.SQLConf".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/core/src/main/scala/org/apache/spark/sql/SparkSessionExtensions.scala:228: Could not find any member to link for "org.apache.spark.sql.catalyst.analysis.FunctionRegistry".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/core/src/main/scala/org/apache/spark/sql/SparkSessionExtensions.scala:190: Could not find any member to link for "org.apache.spark.sql.execution.SparkPlan".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/core/src/main/scala/org/apache/spark/sql/functions.scala:2613: Could not find any member to link for "IllegalArgumentException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/core/src/main/scala/org/apache/spark/sql/functions.scala:2845: Could not find any member to link for "java.time.format.DateTimeFormatter".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/core/src/main/scala/org/apache/spark/sql/functions.scala:3862: The link target "DataFrameReader#csv" is ambiguous. Several members fit the target:
[warn] (paths: String*): org.apache.spark.sql.DataFrame in class DataFrameReader [chosen]
[warn] (csvDataset: org.apache.spark.sql.Dataset[String]): org.apache.spark.sql.DataFrame in class DataFrameReader
[warn] (path: String): org.apache.spark.sql.DataFrame in class DataFrameReader
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/core/src/main/scala/org/apache/spark/sql/functions.scala:3591: The link target "DataFrameReader#json" is ambiguous. Several members fit the target:
[warn] (jsonDataset: org.apache.spark.sql.Dataset[String]): org.apache.spark.sql.DataFrame in class DataFrameReader [chosen]
[warn] (jsonRDD: org.apache.spark.rdd.RDD[String]): org.apache.spark.sql.DataFrame in class DataFrameReader
[warn] (jsonRDD: org.apache.spark.api.java.JavaRDD[String]): org.apache.spark.sql.DataFrame in class DataFrameReader
[warn] (paths: String*): org.apache.spark.sql.DataFrame in class DataFrameReader
[warn] (path: String): org.apache.spark.sql.DataFrame in class DataFrameReader
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/core/src/main/scala/org/apache/spark/sql/functions.scala:2944: Could not find any member to link for "java.time.format.DateTimeFormatter".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/core/src/main/scala/org/apache/spark/sql/functions.scala:2919: Could not find any member to link for "java.time.format.DateTimeFormatter".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/sql/core/src/main/scala/org/apache/spark/sql/functions.scala:2891: Could not find any member to link for "java.time.format.DateTimeFormatter".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/internal/io/HadoopWriteConfigUtil.scala:26: Could not find any member to link for "IllegalArgumentException".
[warn] /**
[warn] ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/FutureAction.scala:56: Could not find any member to link for "Exception".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:442: Could not find any member to link for "IllegalArgumentException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:434: Could not find any member to link for "NumberFormatException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:426: Could not find any member to link for "NumberFormatException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:418: Could not find any member to link for "NumberFormatException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:386: Could not find any member to link for "NumberFormatException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:376: Could not find any member to link for "java.util.NoSuchElementException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:367: Could not find any member to link for "NumberFormatException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:357: Could not find any member to link for "java.util.NoSuchElementException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:348: Could not find any member to link for "NumberFormatException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:338: Could not find any member to link for "java.util.NoSuchElementException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:330: Could not find any member to link for "NumberFormatException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:311: Could not find any member to link for "java.util.NoSuchElementException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:321: Could not find any member to link for "NumberFormatException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:302: Could not find any member to link for "NumberFormatException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:292: Could not find any member to link for "java.util.NoSuchElementException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:283: Could not find any member to link for "NumberFormatException".
[warn]   /**
[warn]   ^
[warn] /home/jenkins/workspace/NewSparkPullRequestBuilder/core/src/main/scala/org/apache/spark/SparkConf.scala:273: Could not find any member to link for "java.util.NoSuchElementException".
[warn]   /**
[warn]   ^
[warn] 85 warnings found
[info] Main Scala API documentation successful.
[success] Total time: 100 s, completed Jul 17, 2019 6:29:29 AM

========================================================================
Running Spark unit tests
========================================================================
[info] Running Spark tests using SBT with these arguments:  -Phadoop-2.7 -Pkubernetes -Dtest.exclude.tags=org.apache.spark.tags.ExtendedHiveTest,org.apache.spark.tags.ExtendedYarnTest kubernetes/test
Using /usr/java/jdk1.8.0_191 as default JAVA_HOME.
Note, this will be overridden by -java-home if it is set.
[info] Loading project definition from /home/jenkins/workspace/NewSparkPullRequestBuilder/project
[info] Set current project to spark-parent (in build file:/home/jenkins/workspace/NewSparkPullRequestBuilder/)
[warn] Multiple main classes detected.  Run 'show discoveredMainClasses' to see the list
[info] Packaging /home/jenkins/workspace/NewSparkPullRequestBuilder/core/target/scala-2.12/spark-core_2.12-3.0.0-SNAPSHOT.jar ...
[info] Done packaging.
[info] ExecutorPodsPollingSnapshotSourceSuite:
[info] - Items returned by the API should be pushed to the event queue (69 milliseconds)
[info] K8sSubmitOpSuite:
[info] - List app status (16 milliseconds)
[info] - List status for multiple apps with glob (5 milliseconds)
[info] - Kill app (3 milliseconds)
[info] - Kill app with gracePeriod (3 milliseconds)
[info] - Kill multiple apps with glob without gracePeriod (3 milliseconds)
[info] KubernetesDriverBuilderSuite:
[info] - use empty initial pod if template is not specified (535 milliseconds)
[info] - load pod template if specified (91 milliseconds)
[info] - complain about misconfigured pod template (7 milliseconds)
[info] DriverKubernetesCredentialsFeatureStepSuite:
[info] - Don't set any credentials (2 milliseconds)
[info] - Only set credentials that are manually mounted. (3 milliseconds)
[info] - Mount credentials from the submission client as a secret. (37 milliseconds)
[info] ExecutorPodsSnapshotSuite:
[info] - States are interpreted correctly from pod metadata. (17 milliseconds)
[info] - Updates add new pods for non-matching ids and edit existing pods for matching ids (5 milliseconds)
[info] MountSecretsFeatureStepSuite:
[info] - mounts all given secrets (8 milliseconds)
[info] PodTemplateConfigMapStepSuite:
[info] - Do nothing when executor template is not specified (1 millisecond)
[info] - Mounts executor template volume if config specified (22 milliseconds)
[info] ExecutorPodsAllocatorSuite:
[info] - Initially request executors in batches. Do not request another batch if the first has not finished. (14 milliseconds)
[info] - Request executors in batches. Allow another batch to be requested if all pending executors start running. (11 milliseconds)
[info] - When a current batch reaches error states immediately, re-request them on the next batch. (8 milliseconds)
[info] - When an executor is requested but the API does not report it in a reasonable time, retry requesting that executor. (7 milliseconds)
[info] HadoopConfDriverFeatureStepSuite:
[info] - mount hadoop config map if defined (4 milliseconds)
[info] - create hadoop config map if config dir is defined (7 milliseconds)
[info] BasicExecutorFeatureStepSuite:
[info] - test spark resource missing vendor (33 milliseconds)
[info] - test spark resource missing amount (28 milliseconds)
[info] - basic executor pod with resources (44 milliseconds)
[info] - basic executor pod has reasonable defaults (35 milliseconds)
[info] - executor pod hostnames get truncated to 63 characters (29 milliseconds)
[info] - hostname truncation generates valid host names (52 milliseconds)
[info] - classpath and extra java options get translated into environment variables (27 milliseconds)
[info] - test executor pyspark memory (26 milliseconds)
[info] - auth secret propagation (34 milliseconds)
[info] - Auth secret shouldn't propagate if files are loaded. (30 milliseconds)
[info] ExecutorPodsWatchSnapshotSourceSuite:
[info] - Watch events should be pushed to the snapshots store as snapshot updates. (3 milliseconds)
[info] ExecutorPodsSnapshotsStoreSuite:
[info] - Subscribers get notified of events periodically. (8 milliseconds)
[info] - Even without sending events, initially receive an empty buffer. (1 millisecond)
[info] - Replacing the snapshot passes the new snapshot to subscribers. (1 millisecond)
[info] DriverServiceFeatureStepSuite:
[info] - Headless service has a port for the driver RPC, the block manager and driver ui. (4 milliseconds)
[info] - Hostname and ports are set according to the service name. (1 millisecond)
[info] - Ports should resolve to defaults in SparkConf and in the service. (2 milliseconds)
[info] - Long prefixes should switch to using a generated unique name. (12 milliseconds)
[info] - Disallow bind address and driver host to be set explicitly. (2 milliseconds)
[info] ExecutorPodsLifecycleManagerSuite:
[info] - When an executor reaches error states immediately, remove from the scheduler backend. (13 milliseconds)
[info] - Don't remove executors twice from Spark but remove from K8s repeatedly. (3 milliseconds)
[info] - When the scheduler backend lists executor ids that aren't present in the cluster, remove those executors from Spark. (1 millisecond)
[info] - Keep executor pods in k8s if configured. (2 milliseconds)
[info] EnvSecretsFeatureStepSuite:
[info] - sets up all keyRefs (5 milliseconds)
[info] DriverCommandFeatureStepSuite:
[info] - java resource (1 millisecond)
[info] - python resource (2 milliseconds)
[info] - R resource (1 millisecond)
[info] ClientSuite:
[info] - The client should configure the pod using the builder. (11 milliseconds)
[info] - The client should create Kubernetes resources (3 milliseconds)
[info] - Waiting for app completion should stall on the watcher (2 milliseconds)
[info] KubernetesClusterSchedulerBackendSuite:
[info] - Start all components (3 milliseconds)
[info] - Stop all components (7 milliseconds)
[info] - Remove executor (1 millisecond)
[info] - Kill executors (12 milliseconds)
[info] - Request total executors (2 milliseconds)
[info] LocalDirsFeatureStepSuite:
[info] - Resolve to default local dir if neither env nor configuration are set (2 milliseconds)
[info] - Use configured local dirs split on comma if provided. (2 milliseconds)
[info] - Use tmpfs to back default local dir (2 milliseconds)
[info] KubernetesUtilsSuite:
[info] - Selects the given container as spark container. (2 milliseconds)
[info] - Selects the first container if no container name is given. (1 millisecond)
[info] - Falls back to the first container if given container name does not exist. (2 milliseconds)
[info] - constructs spark pod correctly with pod template with no containers (1 millisecond)
[info] KubernetesVolumeUtilsSuite:
[info] - Parses hostPath volumes correctly (2 milliseconds)
[info] - Parses subPath correctly (1 millisecond)
[info] - Parses persistentVolumeClaim volumes correctly (1 millisecond)
[info] - Parses emptyDir volumes correctly (1 millisecond)
[info] - Parses emptyDir volume options can be optional (0 milliseconds)
[info] - Defaults optional readOnly to false (0 milliseconds)
[info] - Fails on missing mount key (1 millisecond)
[info] - Fails on missing option key (1 millisecond)
[info] KubernetesConfSuite:
[info] - Resolve driver labels, annotations, secret mount paths, envs, and memory overhead (4 milliseconds)
[info] - Basic executor translated fields. (1 millisecond)
[info] - Image pull secrets. (1 millisecond)
[info] - Set executor labels, annotations, and secrets (3 milliseconds)
[info] BasicDriverFeatureStepSuite:
[info] - Check the pod respects all configurations from the user. (9 milliseconds)
[info] - Check driver pod respects kubernetes driver request cores (7 milliseconds)
[info] - Check appropriate entrypoint rerouting for various bindings (4 milliseconds)
[info] - memory overhead factor: java (2 milliseconds)
[info] - memory overhead factor: python default (2 milliseconds)
[info] - memory overhead factor: python w/ override (2 milliseconds)
[info] - memory overhead factor: r default (2 milliseconds)
[info] KubernetesExecutorBuilderSuite:
[info] - use empty initial pod if template is not specified (44 milliseconds)
[info] - load pod template if specified (37 milliseconds)
[info] - complain about misconfigured pod template (25 milliseconds)
[info] MountVolumesFeatureStepSuite:
[info] - Mounts hostPath volumes (3 milliseconds)
[info] - Mounts pesistentVolumeClaims (4 milliseconds)
[info] - Mounts emptyDir (4 milliseconds)
[info] - Mounts emptyDir with no options (1 millisecond)
[info] - Mounts multiple volumes (2 milliseconds)
[info] - Mounts subpath on emptyDir (1 millisecond)
[info] - Mounts subpath on persistentVolumeClaims (2 milliseconds)
[info] - Mounts multiple subpaths (6 milliseconds)
[info] KerberosConfDriverFeatureStepSuite:
[info] - mount krb5 config map if defined (54 milliseconds)
[info] - create krb5.conf config map if local config provided (52 milliseconds)
[info] - create keytab secret if client keytab file used (24 milliseconds)
[info] - do nothing if container-local keytab used (15 milliseconds)
[info] - mount delegation tokens if provided (14 milliseconds)
[info] - create delegation tokens if needed (42 milliseconds)
[info] - do nothing if no config and no tokens (28 milliseconds)
[info] ScalaTest
[info] Run completed in 6 seconds, 569 milliseconds.
[info] Total number of tests run: 102
[info] Suites: completed 26, aborted 0
[info] Tests: succeeded 102, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] Passed: Total 102, Failed 0, Errors 0, Passed 102
[success] Total time: 14 s, completed Jul 17, 2019 6:29:52 AM
Attempting to post to Github...
 > Post successful.
Archiving artifacts
Recording test results
Finished: SUCCESS