SuccessConsole Output

Skipping 7,691 KB.. Full Log
P0+vIKc0PTOvWC8xrzgzOT8nv0gvODO3ICfVoyQ3xy+/JNU2Yj/Tagmf50wMjD4M7CWJ6SCJEgYhn6zEskT9nMS8dP3gkqLMvHTriiIGKaihyfl5xfk5qXrOEBpkDgMEMDIxMFQUlDCw2+gXFyTm2QEAI9P8iI4AAAA=] Including org.mortbay.jetty:jetty-util:jar:6.1.26 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-client:jar:2.7.4 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-api:jar:2.7.4 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-common:jar:2.7.4 in the shaded jar.
[INFO] Including javax.xml.bind:jaxb-api:jar:2.2.2 in the shaded jar.
[INFO] Including javax.xml.stream:stax-api:jar:1.0-2 in the shaded jar.
[INFO] Including org.codehaus.jackson:jackson-jaxrs:jar:1.9.13 in the shaded jar.
[INFO] Including org.codehaus.jackson:jackson-xc:jar:1.9.13 in the shaded jar.
[INFO] Including org.tukaani:xz:jar:1.5 in the shaded jar.
[INFO] Including org.codehaus.jackson:jackson-core-asl:jar:1.9.13 in the shaded jar.
[INFO] Including org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 in the shaded jar.
[INFO] Including commons-codec:commons-codec:jar:1.10 in the shaded jar.
[INFO] Including org.spark-project.spark:unused:jar:1.0.0 in the shaded jar.
[WARNING] Discovered module-info.class. Shading will break its strong encapsulation.
[WARNING] Discovered module-info.class. Shading will break its strong encapsulation.
[WARNING] spark-streaming-kinesis-asl_2.12-3.1.0-SNAPSHOT.jar, spark-tags_2.12-3.1.0-SNAPSHOT.jar, unused-1.0.0.jar define 1 overlapping classes: 
[WARNING]   - org.apache.spark.unused.UnusedStubClass
[WARNING] maven-shade-plugin has detected that some class files are
[WARNING] present in two or more JARs. When this happens, only one
[WARNING] single version of the class is copied to the uber jar.
[WARNING] Usually this is not harmful and you can skip these warnings,
[WARNING] otherwise try to manually exclude artifacts based on
[WARNING] mvn dependency:tree -Ddetail=true and the above output.
[WARNING] See http://maven.apache.org/plugins/maven-shade-plugin/
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /home/jenkins/workspace/spark-master-test-python-arm/external/kinesis-asl-assembly/target/spark-streaming-kinesis-asl-assembly_2.12-3.1.0-SNAPSHOT.jar with /home/jenkins/workspace/spark-master-test-python-arm/external/kinesis-asl-assembly/target/spark-streaming-kinesis-asl-assembly_2.12-3.1.0-SNAPSHOT-shaded.jar
[INFO] Dependency-reduced POM written at: /home/jenkins/workspace/spark-master-test-python-arm/external/kinesis-asl-assembly/dependency-reduced-pom.xml
[INFO] 
[INFO] --- maven-source-plugin:3.1.0:jar-no-fork (create-source-jar) @ spark-streaming-kinesis-asl-assembly_2.12 ---
[INFO] Building jar: /home/jenkins/workspace/spark-master-test-python-arm/external/kinesis-asl-assembly/target/spark-streaming-kinesis-asl-assembly_2.12-3.1.0-SNAPSHOT-sources.jar
[INFO] 
[INFO] --- maven-source-plugin:3.1.0:test-jar-no-fork (create-source-jar) @ spark-streaming-kinesis-asl-assembly_2.12 ---
[INFO] Building jar: /home/jenkins/workspace/spark-master-test-python-arm/external/kinesis-asl-assembly/target/spark-streaming-kinesis-asl-assembly_2.12-3.1.0-SNAPSHOT-test-sources.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Spark Project Parent POM 3.1.0-SNAPSHOT:
[INFO] 
[INFO] Spark Project Parent POM ........................... SUCCESS [  3.056 s]
[INFO] Spark Project Tags ................................. SUCCESS [  6.839 s]
[INFO] Spark Project Sketch ............................... SUCCESS [  8.324 s]
[INFO] Spark Project Local DB ............................. SUCCESS [  1.917 s]
[INFO] Spark Project Networking ........................... SUCCESS [  3.801 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [  1.523 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 10.393 s]
[INFO] Spark Project Launcher ............................. SUCCESS [  1.948 s]
[INFO] Spark Project Core ................................. SUCCESS [03:22 min]
[INFO] Spark Project ML Local Library ..................... SUCCESS [ 37.253 s]
[INFO] Spark Project GraphX ............................... SUCCESS [ 49.726 s]
[INFO] Spark Project Streaming ............................ SUCCESS [01:23 min]
[INFO] Spark Project Catalyst ............................. SUCCESS [04:02 min]
[INFO] Spark Project SQL .................................. SUCCESS [06:07 min]
[INFO] Spark Project ML Library ........................... SUCCESS [05:05 min]
[INFO] Spark Project Tools ................................ SUCCESS [ 14.551 s]
[INFO] Spark Project Hive ................................. SUCCESS [03:11 min]
[INFO] Spark Project REPL ................................. SUCCESS [ 43.181 s]
[INFO] Spark Project YARN Shuffle Service ................. SUCCESS [ 10.283 s]
[INFO] Spark Project YARN ................................. SUCCESS [01:45 min]
[INFO] Spark Project Mesos ................................ SUCCESS [01:31 min]
[INFO] Spark Project Hive Thrift Server ................... SUCCESS [01:40 min]
[INFO] Spark Project Assembly ............................. SUCCESS [  3.649 s]
[INFO] Kafka 0.10+ Token Provider for Streaming ........... SUCCESS [ 46.875 s]
[INFO] Spark Integration for Kafka 0.10 ................... SUCCESS [01:04 min]
[INFO] Kafka 0.10+ Source for Structured Streaming ........ SUCCESS [01:56 min]
[INFO] Spark Kinesis Integration .......................... SUCCESS [01:07 min]
[INFO] Spark Project Examples ............................. SUCCESS [01:24 min]
[INFO] Spark Integration for Kafka 0.10 Assembly .......... SUCCESS [  5.662 s]
[INFO] Spark Avro ......................................... SUCCESS [01:34 min]
[INFO] Spark Project Kinesis Assembly ..................... SUCCESS [ 10.314 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  39:37 min
[INFO] Finished at: 2020-07-12T06:25:11+08:00
[INFO] ------------------------------------------------------------------------
/home/jenkins/workspace/spark-master-test-python-arm/build/zinc-0.3.15/bin/nailgun: line 50: /home/jenkins/workspace/spark-master-test-python-arm/build/zinc-0.3.15/bin/ng/linux32/ng: cannot execute binary file: Exec format error
+ echo '=========== PySpark test ==========='
=========== PySpark test ===========
+ source /home/jenkins/venv-test/py36/bin/activate
++ '[' /home/jenkins/venv-test/py36/bin/activate = /tmp/hudson8087916052364017760.sh ']'
++ deactivate nondestructive
++ unset -f pydoc
++ '[' -z '' ']'
++ '[' -z '' ']'
++ '[' -n /bin/bash ']'
++ hash -r
++ '[' -z '' ']'
++ unset VIRTUAL_ENV
++ '[' '!' nondestructive = nondestructive ']'
++ VIRTUAL_ENV=/home/jenkins/venv/py36
++ export VIRTUAL_ENV
++ _OLD_VIRTUAL_PATH=/usr/lib/jvm/java-1.8.0-openjdk-arm64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
++ PATH=/home/jenkins/venv/py36/bin:/usr/lib/jvm/java-1.8.0-openjdk-arm64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
++ export PATH
++ '[' -z '' ']'
++ '[' -z '' ']'
++ _OLD_VIRTUAL_PS1=
++ '[' x '!=' x ']'
+++ basename /home/jenkins/venv/py36
++ PS1='(py36) '
++ export PS1
++ alias pydoc
++ true
++ '[' -n /bin/bash ']'
++ hash -r
+ ./python/run-tests --python-executables=python3.6
Running PySpark tests. Output is in /home/jenkins/workspace/spark-master-test-python-arm/python/unit-tests.log
Will test against the following Python executables: ['python3.6']
Will test the following Python modules: ['pyspark-core', 'pyspark-ml', 'pyspark-mllib', 'pyspark-resource', 'pyspark-sql', 'pyspark-streaming']
python3.6 python_implementation is CPython
python3.6 version is: Python 3.6.8
Starting test(python3.6): pyspark.ml.tests.test_algorithms
Starting test(python3.6): pyspark.ml.tests.test_feature
Starting test(python3.6): pyspark.ml.tests.test_base
Starting test(python3.6): pyspark.ml.tests.test_evaluation
Finished test(python3.6): pyspark.ml.tests.test_base (19s)
Starting test(python3.6): pyspark.ml.tests.test_image
Finished test(python3.6): pyspark.ml.tests.test_evaluation (24s)
Starting test(python3.6): pyspark.ml.tests.test_linalg
Finished test(python3.6): pyspark.ml.tests.test_image (24s)
Starting test(python3.6): pyspark.ml.tests.test_param
Finished test(python3.6): pyspark.ml.tests.test_feature (44s)
Starting test(python3.6): pyspark.ml.tests.test_persistence
Finished test(python3.6): pyspark.ml.tests.test_param (27s)
Starting test(python3.6): pyspark.ml.tests.test_pipeline
Finished test(python3.6): pyspark.ml.tests.test_linalg (46s)
Starting test(python3.6): pyspark.ml.tests.test_stat
Finished test(python3.6): pyspark.ml.tests.test_pipeline (7s)
Starting test(python3.6): pyspark.ml.tests.test_training_summary
Finished test(python3.6): pyspark.ml.tests.test_stat (19s)
Starting test(python3.6): pyspark.ml.tests.test_tuning
Finished test(python3.6): pyspark.ml.tests.test_persistence (74s)
Starting test(python3.6): pyspark.ml.tests.test_wrapper
Finished test(python3.6): pyspark.ml.tests.test_algorithms (120s)
Starting test(python3.6): pyspark.mllib.tests.test_algorithms
Finished test(python3.6): pyspark.ml.tests.test_training_summary (47s)
Starting test(python3.6): pyspark.mllib.tests.test_feature
Finished test(python3.6): pyspark.ml.tests.test_wrapper (22s)
Starting test(python3.6): pyspark.mllib.tests.test_linalg
Finished test(python3.6): pyspark.mllib.tests.test_feature (41s)
Starting test(python3.6): pyspark.mllib.tests.test_stat
Finished test(python3.6): pyspark.mllib.tests.test_stat (33s)
Starting test(python3.6): pyspark.mllib.tests.test_streaming_algorithms
Finished test(python3.6): pyspark.mllib.tests.test_algorithms (86s)
Starting test(python3.6): pyspark.mllib.tests.test_util
Finished test(python3.6): pyspark.mllib.tests.test_util (14s)
Starting test(python3.6): pyspark.sql.tests.test_arrow
Finished test(python3.6): pyspark.mllib.tests.test_linalg (81s)
Starting test(python3.6): pyspark.sql.tests.test_catalog
Finished test(python3.6): pyspark.sql.tests.test_arrow (0s) ... 57 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_column
Finished test(python3.6): pyspark.sql.tests.test_catalog (23s)
Starting test(python3.6): pyspark.sql.tests.test_conf
Finished test(python3.6): pyspark.sql.tests.test_column (24s)
Starting test(python3.6): pyspark.sql.tests.test_context
Finished test(python3.6): pyspark.sql.tests.test_conf (11s)
Starting test(python3.6): pyspark.sql.tests.test_dataframe
Finished test(python3.6): pyspark.sql.tests.test_context (35s)
Starting test(python3.6): pyspark.sql.tests.test_datasources
Finished test(python3.6): pyspark.sql.tests.test_datasources (26s)
Starting test(python3.6): pyspark.sql.tests.test_functions
Finished test(python3.6): pyspark.sql.tests.test_dataframe (61s) ... 3 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_group
Finished test(python3.6): pyspark.sql.tests.test_group (23s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_cogrouped_map
Finished test(python3.6): pyspark.sql.tests.test_pandas_cogrouped_map (0s) ... 15 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_grouped_map
Finished test(python3.6): pyspark.sql.tests.test_pandas_grouped_map (0s) ... 21 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_map
Finished test(python3.6): pyspark.sql.tests.test_pandas_map (0s) ... 6 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf (0s) ... 6 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_grouped_agg
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_grouped_agg (0s) ... 16 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_scalar
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_scalar (0s) ... 50 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_typehints
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_typehints (0s) ... 10 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_window
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_window (0s) ... 14 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_readwriter
Finished test(python3.6): pyspark.sql.tests.test_functions (40s)
Starting test(python3.6): pyspark.sql.tests.test_serde
Finished test(python3.6): pyspark.ml.tests.test_tuning (258s)
Starting test(python3.6): pyspark.sql.tests.test_session
Finished test(python3.6): pyspark.sql.tests.test_serde (27s)
Starting test(python3.6): pyspark.sql.tests.test_streaming
Finished test(python3.6): pyspark.sql.tests.test_readwriter (32s)
Starting test(python3.6): pyspark.sql.tests.test_types
Finished test(python3.6): pyspark.mllib.tests.test_streaming_algorithms (181s)
Starting test(python3.6): pyspark.sql.tests.test_udf
Finished test(python3.6): pyspark.sql.tests.test_session (40s)
Starting test(python3.6): pyspark.sql.tests.test_utils
Finished test(python3.6): pyspark.sql.tests.test_utils (14s)
Starting test(python3.6): pyspark.streaming.tests.test_context
Finished test(python3.6): pyspark.sql.tests.test_streaming (44s)
Starting test(python3.6): pyspark.streaming.tests.test_dstream
Finished test(python3.6): pyspark.streaming.tests.test_context (27s)
Starting test(python3.6): pyspark.streaming.tests.test_kinesis
Finished test(python3.6): pyspark.streaming.tests.test_kinesis (0s) ... 2 tests were skipped
Starting test(python3.6): pyspark.streaming.tests.test_listener
Finished test(python3.6): pyspark.sql.tests.test_types (58s)
Starting test(python3.6): pyspark.tests.test_appsubmit
Finished test(python3.6): pyspark.streaming.tests.test_listener (12s)
Starting test(python3.6): pyspark.tests.test_broadcast
Finished test(python3.6): pyspark.sql.tests.test_udf (69s)
Starting test(python3.6): pyspark.tests.test_conf
Finished test(python3.6): pyspark.tests.test_conf (20s)
Starting test(python3.6): pyspark.tests.test_context
Finished test(python3.6): pyspark.tests.test_broadcast (42s)
Starting test(python3.6): pyspark.tests.test_daemon
Finished test(python3.6): pyspark.tests.test_daemon (5s)
Starting test(python3.6): pyspark.tests.test_join
Finished test(python3.6): pyspark.tests.test_join (14s)
Starting test(python3.6): pyspark.tests.test_profiler
Finished test(python3.6): pyspark.tests.test_context (47s)
Starting test(python3.6): pyspark.tests.test_rdd
Finished test(python3.6): pyspark.tests.test_profiler (12s)
Starting test(python3.6): pyspark.tests.test_rddbarrier
Finished test(python3.6): pyspark.tests.test_rddbarrier (10s)
Starting test(python3.6): pyspark.tests.test_readwrite
Finished test(python3.6): pyspark.streaming.tests.test_dstream (124s)
Starting test(python3.6): pyspark.tests.test_serializers
Finished test(python3.6): pyspark.tests.test_appsubmit (106s)
Starting test(python3.6): pyspark.tests.test_shuffle
Finished test(python3.6): pyspark.tests.test_readwrite (26s) ... 3 tests were skipped
Starting test(python3.6): pyspark.tests.test_taskcontext
Finished test(python3.6): pyspark.tests.test_serializers (13s)
Starting test(python3.6): pyspark.tests.test_util
Finished test(python3.6): pyspark.tests.test_shuffle (13s)
Starting test(python3.6): pyspark.tests.test_worker
Finished test(python3.6): pyspark.tests.test_util (6s)
Starting test(python3.6): pyspark.accumulators
Finished test(python3.6): pyspark.accumulators (7s)
Starting test(python3.6): pyspark.broadcast
Finished test(python3.6): pyspark.tests.test_worker (19s)
Starting test(python3.6): pyspark.conf
Finished test(python3.6): pyspark.broadcast (6s)
Starting test(python3.6): pyspark.context
Finished test(python3.6): pyspark.conf (5s)
Starting test(python3.6): pyspark.ml.classification
Finished test(python3.6): pyspark.context (24s)
Starting test(python3.6): pyspark.ml.clustering
Finished test(python3.6): pyspark.tests.test_taskcontext (71s)
Starting test(python3.6): pyspark.ml.evaluation
Finished test(python3.6): pyspark.ml.clustering (48s)
Starting test(python3.6): pyspark.ml.feature
Finished test(python3.6): pyspark.tests.test_rdd (135s)
Starting test(python3.6): pyspark.ml.fpm
Finished test(python3.6): pyspark.ml.evaluation (29s)
Starting test(python3.6): pyspark.ml.functions
Finished test(python3.6): pyspark.ml.classification (85s)
Starting test(python3.6): pyspark.ml.image
Finished test(python3.6): pyspark.ml.functions (17s)
Starting test(python3.6): pyspark.ml.linalg.__init__
Finished test(python3.6): pyspark.ml.linalg.__init__ (0s)
Starting test(python3.6): pyspark.ml.recommendation
Finished test(python3.6): pyspark.ml.image (10s)
Starting test(python3.6): pyspark.ml.regression
Finished test(python3.6): pyspark.ml.fpm (22s)
Starting test(python3.6): pyspark.ml.stat
Finished test(python3.6): pyspark.ml.stat (37s)
Starting test(python3.6): pyspark.ml.tuning
Finished test(python3.6): pyspark.ml.regression (64s)
Starting test(python3.6): pyspark.mllib.classification
Finished test(python3.6): pyspark.ml.tuning (44s)
Starting test(python3.6): pyspark.mllib.clustering
Finished test(python3.6): pyspark.ml.feature (124s)
Starting test(python3.6): pyspark.mllib.evaluation
Finished test(python3.6): pyspark.mllib.classification (45s)
Starting test(python3.6): pyspark.mllib.feature
Finished test(python3.6): pyspark.ml.recommendation (122s)
Starting test(python3.6): pyspark.mllib.fpm
Finished test(python3.6): pyspark.mllib.evaluation (27s)
Starting test(python3.6): pyspark.mllib.linalg.__init__
Finished test(python3.6): pyspark.mllib.linalg.__init__ (0s)
Starting test(python3.6): pyspark.mllib.linalg.distributed
Finished test(python3.6): pyspark.mllib.feature (34s)
Starting test(python3.6): pyspark.mllib.random
Finished test(python3.6): pyspark.mllib.clustering (63s)
Starting test(python3.6): pyspark.mllib.recommendation
Finished test(python3.6): pyspark.mllib.fpm (26s)
Starting test(python3.6): pyspark.mllib.regression
Finished test(python3.6): pyspark.mllib.random (15s)
Starting test(python3.6): pyspark.mllib.stat.KernelDensity
Finished test(python3.6): pyspark.mllib.stat.KernelDensity (0s)
Starting test(python3.6): pyspark.mllib.stat._statistics
Finished test(python3.6): pyspark.mllib.stat._statistics (22s)
Starting test(python3.6): pyspark.mllib.tree
Finished test(python3.6): pyspark.mllib.regression (36s)
Starting test(python3.6): pyspark.mllib.util
Finished test(python3.6): pyspark.mllib.recommendation (48s)
Starting test(python3.6): pyspark.profiler
Finished test(python3.6): pyspark.mllib.linalg.distributed (69s)
Starting test(python3.6): pyspark.rdd
Finished test(python3.6): pyspark.profiler (10s)
Starting test(python3.6): pyspark.resource.tests.test_resources
Finished test(python3.6): pyspark.mllib.tree (24s)
Starting test(python3.6): pyspark.serializers
Finished test(python3.6): pyspark.mllib.util (23s)
Starting test(python3.6): pyspark.shuffle
Finished test(python3.6): pyspark.shuffle (0s)
Starting test(python3.6): pyspark.sql.avro.functions
Finished test(python3.6): pyspark.resource.tests.test_resources (7s)
Starting test(python3.6): pyspark.sql.catalog
Finished test(python3.6): pyspark.serializers (12s)
Starting test(python3.6): pyspark.sql.column
Finished test(python3.6): pyspark.sql.avro.functions (17s)
Starting test(python3.6): pyspark.sql.conf
Finished test(python3.6): pyspark.sql.catalog (16s)
Starting test(python3.6): pyspark.sql.context
Finished test(python3.6): pyspark.sql.conf (7s)
Starting test(python3.6): pyspark.sql.dataframe
Finished test(python3.6): pyspark.rdd (44s)
Starting test(python3.6): pyspark.sql.functions
Finished test(python3.6): pyspark.sql.column (33s)
Starting test(python3.6): pyspark.sql.group
Finished test(python3.6): pyspark.sql.context (24s)
Starting test(python3.6): pyspark.sql.pandas.conversion
Finished test(python3.6): pyspark.sql.pandas.conversion (10s)
Starting test(python3.6): pyspark.sql.pandas.group_ops
Finished test(python3.6): pyspark.sql.pandas.group_ops (17s)
Starting test(python3.6): pyspark.sql.pandas.map_ops
Finished test(python3.6): pyspark.sql.group (40s)
Starting test(python3.6): pyspark.sql.pandas.serializers
Finished test(python3.6): pyspark.sql.pandas.serializers (0s)
Starting test(python3.6): pyspark.sql.pandas.typehints
Finished test(python3.6): pyspark.sql.pandas.typehints (0s)
Starting test(python3.6): pyspark.sql.pandas.types
Finished test(python3.6): pyspark.sql.pandas.types (0s)
Starting test(python3.6): pyspark.sql.pandas.utils
Finished test(python3.6): pyspark.sql.pandas.utils (0s)
Starting test(python3.6): pyspark.sql.readwriter
Finished test(python3.6): pyspark.sql.pandas.map_ops (16s)
Starting test(python3.6): pyspark.sql.session
Finished test(python3.6): pyspark.sql.dataframe (82s)
Starting test(python3.6): pyspark.sql.streaming
Finished test(python3.6): pyspark.sql.functions (80s)
Starting test(python3.6): pyspark.sql.types
Finished test(python3.6): pyspark.sql.readwriter (27s)
Starting test(python3.6): pyspark.sql.udf
Finished test(python3.6): pyspark.sql.session (25s)
Starting test(python3.6): pyspark.sql.window
Finished test(python3.6): pyspark.sql.types (8s)
Starting test(python3.6): pyspark.streaming.util
Finished test(python3.6): pyspark.streaming.util (0s)
Starting test(python3.6): pyspark.util
Finished test(python3.6): pyspark.util (0s)
Finished test(python3.6): pyspark.sql.udf (11s)
Finished test(python3.6): pyspark.sql.streaming (17s)
Finished test(python3.6): pyspark.sql.window (17s)
Tests passed in 1019 seconds

Skipped tests in pyspark.sql.tests.test_arrow with python3.6:
    test_createDataFrame_column_name_encoding (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_does_not_modify_input (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_fallback_disabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_fallback_enabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_respect_session_timezone (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_toggle (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_array_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_float_index (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_incorrect_schema (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_int_col_names (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_names (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_schema (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_single_data_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDateFrame_with_category_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_filtered_frame (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_no_partition_frame (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_null_conversion (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_pandas_round_trip (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_propagates_spark_exception (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_schema_conversion_roundtrip (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_timestamp_dst (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_timestamp_nat (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_arrow_toggle (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_batch_order (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_fallback_disabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_fallback_enabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_respect_session_timezone (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_with_array_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_column_name_encoding (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_does_not_modify_input (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_fallback_disabled (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_fallback_enabled (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_respect_session_timezone (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_toggle (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_array_type (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_float_index (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_incorrect_schema (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_int_col_names (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_names (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_schema (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_single_data_type (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDateFrame_with_category_type (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_filtered_frame (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_no_partition_frame (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_null_conversion (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_pandas_round_trip (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_propagates_spark_exception (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_schema_conversion_roundtrip (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_timestamp_dst (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_timestamp_nat (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_arrow_toggle (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_batch_order (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_fallback_disabled (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_fallback_enabled (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_respect_session_timezone (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_with_array_type (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_exception_by_max_results (pyspark.sql.tests.test_arrow.MaxResultArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_dataframe with python3.6:
    test_create_dataframe_required_pandas_not_found (pyspark.sql.tests.test_dataframe.DataFrameTests) ... skipped 'Required Pandas was found.'
    test_to_pandas_required_pandas_not_found (pyspark.sql.tests.test_dataframe.DataFrameTests) ... skipped 'Required Pandas was found.'
    test_query_execution_listener_on_collect_with_arrow (pyspark.sql.tests.test_dataframe.QueryExecutionListenerTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_cogrouped_map with python3.6:
    test_case_insensitive_grouping_column (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_complex_group_by (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_different_schemas (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_empty_group_by (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_left_group_empty (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_scalar_udfs_followed_by_cogrouby_apply (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_right_group_empty (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_simple (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_with_key_complex (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_with_key_left (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_with_key_left_group_empty (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_with_key_right (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_with_key_right_group_empty (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_wrong_args (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_wrong_return_type (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_grouped_map with python3.6:
    test_array_type_correct (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_case_insensitive_grouping_column (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_coerce (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_column_order (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_complex_groupby (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_datatype_string (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_decorator (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_empty_groupby (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_grouped_over_window (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_grouped_over_window_with_key (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_grouped_with_empty_partition (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_scalar_udfs_followed_by_grouby_apply (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_positional_assignment_conf (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_register_grouped_map_udf (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_self_join_with_pandas (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_supported_types (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_timestamp_dst (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_udf_with_key (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_unsupported_types (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_wrong_args (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_wrong_return_type (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_map with python3.6:
    test_chain_map_partitions_in_pandas (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_different_output_length (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_empty_iterator (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_empty_rows (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_map_partitions_in_pandas (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_multiple_columns (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_udf with python3.6:
    test_pandas_udf_arrow_overflow (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_pandas_udf_basic (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_pandas_udf_decorator (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_pandas_udf_detect_unsafe_type_conversion (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_stopiteration_in_udf (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_udf_wrong_arg (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_udf_grouped_agg with python3.6:
    test_alias (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_array_type (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_basic (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_complex_expressions (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_complex_groupby (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_grouped_with_empty_partition (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_grouped_without_group_by_clause (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_invalid_args (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_manual (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_sql (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_udfs (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_multiple_udfs (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_no_predicate_pushdown_through (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_register_vectorized_udf_basic (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_retain_group_columns (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_unsupported_types (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_udf_scalar with python3.6:
    test_datasource_with_udf (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_udf (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_udf_and_sql (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_nondeterministic_vectorized_udf (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_nondeterministic_vectorized_udf_in_aggregate (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_pandas_udf_nested_arrays (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_pandas_udf_tokenize (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_register_nondeterministic_vectorized_udf_basic (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_register_vectorized_udf_basic (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_scalar_iter_udf_close (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_scalar_iter_udf_close_early (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_scalar_iter_udf_init (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_timestamp_dst (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_type_annotation (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_udf_category_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_array_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_basic (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_chained (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_chained_struct_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_check_config (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_complex (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_datatype_string (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_dates (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_decorator (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_empty_partition (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_exception (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_invalid_length (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_nested_struct (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_array (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_binary (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_boolean (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_byte (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_decimal (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_double (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_float (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_int (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_long (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_short (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_string (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_return_scalar (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_return_timestamp_tz (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_string_in_udf (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_struct_complex (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_struct_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_struct_with_empty_partition (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_timestamps (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_timestamps_respect_session_timezone (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_unsupported_types (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_varargs (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_wrong_return_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_udf_typehints with python3.6:
    test_group_agg_udf_type_hint (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_ignore_type_hint_in_cogroup_apply_in_pandas (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_ignore_type_hint_in_group_apply_in_pandas (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_ignore_type_hint_in_map_in_pandas (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_scalar_iter_udf_type_hint (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_scalar_udf_type_hint (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_type_annotation_group_agg (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_type_annotation_negative (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_type_annotation_scalar (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_type_annotation_scalar_iter (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_udf_window with python3.6:
    test_array_type (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_bounded_mixed (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_bounded_simple (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_growing_window (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_invalid_args (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_sql (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_sql_and_udf (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_udf (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_multiple_udfs (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_replace_existing (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_shrinking_window (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_simple (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_sliding_window (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_without_partitionBy (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.streaming.tests.test_kinesis with python3.6:
    test_kinesis_stream (pyspark.streaming.tests.test_kinesis.KinesisStreamTests) ... skipped "Skipping all Kinesis Python tests as environmental variable 'ENABLE_KINESIS_TESTS' was not set."
    test_kinesis_stream_api (pyspark.streaming.tests.test_kinesis.KinesisStreamTests) ... skipped "Skipping all Kinesis Python tests as environmental variable 'ENABLE_KINESIS_TESTS' was not set."

Skipped tests in pyspark.tests.test_readwrite with python3.6:
    test_sequencefiles (pyspark.tests.test_readwrite.InputFormatTests) ... skipped 'serialize array of byte'
    test_newhadoop_with_array (pyspark.tests.test_readwrite.OutputFormatTests) ... skipped 'serialize of array'
    test_sequencefiles (pyspark.tests.test_readwrite.OutputFormatTests) ... skipped 'serialize array of byte'
Archiving artifacts
Finished: SUCCESS