SuccessConsole Output

Skipping 7,587 KB.. Full Log
gYhn6zEskT9nMS8dP3gkqLMvHTriiIGKaihyfl5xfk5qXrOEBpkDgMEMDIxMFQUlDDI2RQXJOYpFJdU5qTaKoEttlJQNnS1NHBzs1ayAwAGsmNKpQAAAA==INFO] Including commons-io:commons-io:jar:2.4 in the shaded jar.
[INFO] Including org.apache.commons:commons-compress:jar:1.4.1 in the shaded jar.
[INFO] Including org.mortbay.jetty:jetty-util:jar:6.1.26 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-client:jar:2.7.4 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-api:jar:2.7.4 in the shaded jar.
[INFO] Including org.apache.hadoop:hadoop-yarn-common:jar:2.7.4 in the shaded jar.
[INFO] Including javax.xml.bind:jaxb-api:jar:2.2.2 in the shaded jar.
[INFO] Including javax.xml.stream:stax-api:jar:1.0-2 in the shaded jar.
[INFO] Including org.codehaus.jackson:jackson-jaxrs:jar:1.9.13 in the shaded jar.
[INFO] Including org.codehaus.jackson:jackson-xc:jar:1.9.13 in the shaded jar.
[INFO] Including org.tukaani:xz:jar:1.5 in the shaded jar.
[INFO] Including org.codehaus.jackson:jackson-core-asl:jar:1.9.13 in the shaded jar.
[INFO] Including org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13 in the shaded jar.
[INFO] Including commons-codec:commons-codec:jar:1.10 in the shaded jar.
[INFO] Including org.spark-project.spark:unused:jar:1.0.0 in the shaded jar.
[WARNING] Discovered module-info.class. Shading will break its strong encapsulation.
[WARNING] Discovered module-info.class. Shading will break its strong encapsulation.
[WARNING] spark-streaming-kinesis-asl_2.12-3.0.0-SNAPSHOT.jar, spark-tags_2.12-3.0.0-SNAPSHOT.jar, unused-1.0.0.jar define 1 overlapping classes: 
[WARNING]   - org.apache.spark.unused.UnusedStubClass
[WARNING] maven-shade-plugin has detected that some class files are
[WARNING] present in two or more JARs. When this happens, only one
[WARNING] single version of the class is copied to the uber jar.
[WARNING] Usually this is not harmful and you can skip these warnings,
[WARNING] otherwise try to manually exclude artifacts based on
[WARNING] mvn dependency:tree -Ddetail=true and the above output.
[WARNING] See http://maven.apache.org/plugins/maven-shade-plugin/
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing /home/jenkins/workspace/spark-master-test-python-arm/external/kinesis-asl-assembly/target/spark-streaming-kinesis-asl-assembly_2.12-3.0.0-SNAPSHOT.jar with /home/jenkins/workspace/spark-master-test-python-arm/external/kinesis-asl-assembly/target/spark-streaming-kinesis-asl-assembly_2.12-3.0.0-SNAPSHOT-shaded.jar
[INFO] Dependency-reduced POM written at: /home/jenkins/workspace/spark-master-test-python-arm/external/kinesis-asl-assembly/dependency-reduced-pom.xml
[INFO] 
[INFO] --- maven-source-plugin:3.1.0:jar-no-fork (create-source-jar) @ spark-streaming-kinesis-asl-assembly_2.12 ---
[INFO] Building jar: /home/jenkins/workspace/spark-master-test-python-arm/external/kinesis-asl-assembly/target/spark-streaming-kinesis-asl-assembly_2.12-3.0.0-SNAPSHOT-sources.jar
[INFO] 
[INFO] --- maven-source-plugin:3.1.0:test-jar-no-fork (create-source-jar) @ spark-streaming-kinesis-asl-assembly_2.12 ---
[INFO] Building jar: /home/jenkins/workspace/spark-master-test-python-arm/external/kinesis-asl-assembly/target/spark-streaming-kinesis-asl-assembly_2.12-3.0.0-SNAPSHOT-test-sources.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Spark Project Parent POM 3.0.0-SNAPSHOT:
[INFO] 
[INFO] Spark Project Parent POM ........................... SUCCESS [  5.282 s]
[INFO] Spark Project Tags ................................. SUCCESS [ 10.579 s]
[INFO] Spark Project Sketch ............................... SUCCESS [ 10.798 s]
[INFO] Spark Project Local DB ............................. SUCCESS [  1.557 s]
[INFO] Spark Project Networking ........................... SUCCESS [  3.449 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [  1.230 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [  9.594 s]
[INFO] Spark Project Launcher ............................. SUCCESS [  1.577 s]
[INFO] Spark Project Core ................................. SUCCESS [02:53 min]
[INFO] Spark Project ML Local Library ..................... SUCCESS [ 36.971 s]
[INFO] Spark Project GraphX ............................... SUCCESS [ 47.453 s]
[INFO] Spark Project Streaming ............................ SUCCESS [01:22 min]
[INFO] Spark Project Catalyst ............................. SUCCESS [03:57 min]
[INFO] Spark Project SQL .................................. SUCCESS [05:50 min]
[INFO] Spark Project ML Library ........................... SUCCESS [04:54 min]
[INFO] Spark Project Tools ................................ SUCCESS [ 14.483 s]
[INFO] Spark Project Hive ................................. SUCCESS [03:08 min]
[INFO] Spark Project REPL ................................. SUCCESS [ 41.174 s]
[INFO] Spark Project YARN Shuffle Service ................. SUCCESS [ 10.250 s]
[INFO] Spark Project YARN ................................. SUCCESS [01:39 min]
[INFO] Spark Project Mesos ................................ SUCCESS [01:32 min]
[INFO] Spark Project Hive Thrift Server ................... SUCCESS [01:35 min]
[INFO] Spark Project Assembly ............................. SUCCESS [  3.499 s]
[INFO] Kafka 0.10+ Token Provider for Streaming ........... SUCCESS [ 47.420 s]
[INFO] Spark Integration for Kafka 0.10 ................... SUCCESS [01:05 min]
[INFO] Kafka 0.10+ Source for Structured Streaming ........ SUCCESS [01:53 min]
[INFO] Spark Kinesis Integration .......................... SUCCESS [01:09 min]
[INFO] Spark Project Examples ............................. SUCCESS [01:26 min]
[INFO] Spark Integration for Kafka 0.10 Assembly .......... SUCCESS [  5.580 s]
[INFO] Spark Avro ......................................... SUCCESS [01:28 min]
[INFO] Spark Project Kinesis Assembly ..................... SUCCESS [  9.566 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  38:19 min
[INFO] Finished at: 2020-01-25T09:26:54+08:00
[INFO] ------------------------------------------------------------------------
/home/jenkins/workspace/spark-master-test-python-arm/build/zinc-0.3.15/bin/nailgun: line 50: /home/jenkins/workspace/spark-master-test-python-arm/build/zinc-0.3.15/bin/ng/linux32/ng: cannot execute binary file: Exec format error
+ echo '=========== PySpark test ==========='
=========== PySpark test ===========
+ source /home/jenkins/venv-test/py36/bin/activate
++ '[' /home/jenkins/venv-test/py36/bin/activate = /tmp/hudson4027681018642942210.sh ']'
++ deactivate nondestructive
++ unset -f pydoc
++ '[' -z '' ']'
++ '[' -z '' ']'
++ '[' -n /bin/bash ']'
++ hash -r
++ '[' -z '' ']'
++ unset VIRTUAL_ENV
++ '[' '!' nondestructive = nondestructive ']'
++ VIRTUAL_ENV=/home/jenkins/venv/py36
++ export VIRTUAL_ENV
++ _OLD_VIRTUAL_PATH=/usr/lib/jvm/java-1.8.0-openjdk-arm64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
++ PATH=/home/jenkins/venv/py36/bin:/usr/lib/jvm/java-1.8.0-openjdk-arm64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
++ export PATH
++ '[' -z '' ']'
++ '[' -z '' ']'
++ _OLD_VIRTUAL_PS1=
++ '[' x '!=' x ']'
+++ basename /home/jenkins/venv/py36
++ PS1='(py36) '
++ export PS1
++ alias pydoc
++ true
++ '[' -n /bin/bash ']'
++ hash -r
+ ./python/run-tests --python-executables=python3.6
Running PySpark tests. Output is in /home/jenkins/workspace/spark-master-test-python-arm/python/unit-tests.log
Will test against the following Python executables: ['python3.6']
Will test the following Python modules: ['pyspark-core', 'pyspark-ml', 'pyspark-mllib', 'pyspark-sql', 'pyspark-streaming']
python3.6 python_implementation is CPython
python3.6 version is: Python 3.6.8
Starting test(python3.6): pyspark.ml.tests.test_algorithms
Starting test(python3.6): pyspark.ml.tests.test_base
Starting test(python3.6): pyspark.ml.tests.test_evaluation
Starting test(python3.6): pyspark.ml.tests.test_feature
Finished test(python3.6): pyspark.ml.tests.test_base (18s)
Starting test(python3.6): pyspark.ml.tests.test_image
Finished test(python3.6): pyspark.ml.tests.test_evaluation (23s)
Starting test(python3.6): pyspark.ml.tests.test_linalg
Finished test(python3.6): pyspark.ml.tests.test_image (21s)
Starting test(python3.6): pyspark.ml.tests.test_param
Finished test(python3.6): pyspark.ml.tests.test_feature (41s)
Starting test(python3.6): pyspark.ml.tests.test_persistence
Finished test(python3.6): pyspark.ml.tests.test_param (25s)
Starting test(python3.6): pyspark.ml.tests.test_pipeline
Finished test(python3.6): pyspark.ml.tests.test_linalg (43s)
Starting test(python3.6): pyspark.ml.tests.test_stat
Finished test(python3.6): pyspark.ml.tests.test_pipeline (6s)
Starting test(python3.6): pyspark.ml.tests.test_training_summary
Finished test(python3.6): pyspark.ml.tests.test_stat (19s)
Starting test(python3.6): pyspark.ml.tests.test_tuning
Finished test(python3.6): pyspark.ml.tests.test_persistence (56s)
Starting test(python3.6): pyspark.ml.tests.test_wrapper
Finished test(python3.6): pyspark.ml.tests.test_training_summary (38s)
Starting test(python3.6): pyspark.mllib.tests.test_algorithms
Finished test(python3.6): pyspark.ml.tests.test_algorithms (111s)
Starting test(python3.6): pyspark.mllib.tests.test_feature
Finished test(python3.6): pyspark.ml.tests.test_wrapper (21s)
Starting test(python3.6): pyspark.mllib.tests.test_linalg
Finished test(python3.6): pyspark.mllib.tests.test_feature (40s)
Starting test(python3.6): pyspark.mllib.tests.test_stat
Finished test(python3.6): pyspark.mllib.tests.test_stat (32s)
Starting test(python3.6): pyspark.mllib.tests.test_streaming_algorithms
Finished test(python3.6): pyspark.mllib.tests.test_algorithms (84s)
Starting test(python3.6): pyspark.mllib.tests.test_util
Finished test(python3.6): pyspark.mllib.tests.test_linalg (79s)
Starting test(python3.6): pyspark.sql.tests.test_arrow
Finished test(python3.6): pyspark.sql.tests.test_arrow (0s) ... 53 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_catalog
Finished test(python3.6): pyspark.mllib.tests.test_util (14s)
Starting test(python3.6): pyspark.sql.tests.test_column
Finished test(python3.6): pyspark.ml.tests.test_tuning (123s)
Starting test(python3.6): pyspark.sql.tests.test_conf
Finished test(python3.6): pyspark.sql.tests.test_catalog (21s)
Starting test(python3.6): pyspark.sql.tests.test_context
Finished test(python3.6): pyspark.sql.tests.test_conf (12s)
Starting test(python3.6): pyspark.sql.tests.test_dataframe
Finished test(python3.6): pyspark.sql.tests.test_column (23s)
Starting test(python3.6): pyspark.sql.tests.test_datasources
Finished test(python3.6): pyspark.sql.tests.test_context (33s)
Starting test(python3.6): pyspark.sql.tests.test_functions
Finished test(python3.6): pyspark.sql.tests.test_datasources (26s)
Starting test(python3.6): pyspark.sql.tests.test_group
Finished test(python3.6): pyspark.sql.tests.test_dataframe (56s) ... 3 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_cogrouped_map
Finished test(python3.6): pyspark.sql.tests.test_pandas_cogrouped_map (0s) ... 14 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_grouped_map
Finished test(python3.6): pyspark.sql.tests.test_group (22s)
Starting test(python3.6): pyspark.sql.tests.test_pandas_map
Finished test(python3.6): pyspark.sql.tests.test_pandas_grouped_map (0s) ... 20 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf
Finished test(python3.6): pyspark.sql.tests.test_pandas_map (0s) ... 6 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_grouped_agg
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf (0s) ... 6 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_scalar
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_grouped_agg (0s) ... 15 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_typehints
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_scalar (0s) ... 49 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_pandas_udf_window
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_typehints (0s) ... 7 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_readwriter
Finished test(python3.6): pyspark.sql.tests.test_pandas_udf_window (0s) ... 14 tests were skipped
Starting test(python3.6): pyspark.sql.tests.test_serde
Finished test(python3.6): pyspark.sql.tests.test_functions (38s)
Starting test(python3.6): pyspark.sql.tests.test_session
Finished test(python3.6): pyspark.sql.tests.test_serde (26s)
Starting test(python3.6): pyspark.sql.tests.test_streaming
Finished test(python3.6): pyspark.sql.tests.test_readwriter (32s)
Starting test(python3.6): pyspark.sql.tests.test_types
Finished test(python3.6): pyspark.sql.tests.test_session (33s)
Starting test(python3.6): pyspark.sql.tests.test_udf
Finished test(python3.6): pyspark.sql.tests.test_streaming (39s)
Starting test(python3.6): pyspark.sql.tests.test_utils
Finished test(python3.6): pyspark.mllib.tests.test_streaming_algorithms (171s)
Starting test(python3.6): pyspark.streaming.tests.test_context
Finished test(python3.6): pyspark.sql.tests.test_utils (14s)
Starting test(python3.6): pyspark.streaming.tests.test_dstream
Finished test(python3.6): pyspark.sql.tests.test_types (52s)
Starting test(python3.6): pyspark.streaming.tests.test_kinesis
Finished test(python3.6): pyspark.streaming.tests.test_kinesis (0s) ... 2 tests were skipped
Starting test(python3.6): pyspark.streaming.tests.test_listener
Finished test(python3.6): pyspark.streaming.tests.test_listener (11s)
Starting test(python3.6): pyspark.tests.test_appsubmit
Finished test(python3.6): pyspark.streaming.tests.test_context (25s)
Starting test(python3.6): pyspark.tests.test_broadcast
Finished test(python3.6): pyspark.sql.tests.test_udf (63s)
Starting test(python3.6): pyspark.tests.test_conf
Finished test(python3.6): pyspark.tests.test_conf (19s)
Starting test(python3.6): pyspark.tests.test_context
Finished test(python3.6): pyspark.tests.test_broadcast (39s)
Starting test(python3.6): pyspark.tests.test_daemon
Finished test(python3.6): pyspark.tests.test_daemon (5s)
Starting test(python3.6): pyspark.tests.test_join
Finished test(python3.6): pyspark.tests.test_join (11s)
Starting test(python3.6): pyspark.tests.test_profiler
Finished test(python3.6): pyspark.tests.test_context (35s)
Starting test(python3.6): pyspark.tests.test_rdd
Finished test(python3.6): pyspark.tests.test_profiler (12s)
Starting test(python3.6): pyspark.tests.test_rddbarrier
Finished test(python3.6): pyspark.tests.test_rddbarrier (9s)
Starting test(python3.6): pyspark.tests.test_readwrite
Finished test(python3.6): pyspark.tests.test_appsubmit (99s)
Starting test(python3.6): pyspark.tests.test_serializers
Finished test(python3.6): pyspark.tests.test_readwrite (24s) ... 3 tests were skipped
Starting test(python3.6): pyspark.tests.test_shuffle
Finished test(python3.6): pyspark.streaming.tests.test_dstream (123s)
Starting test(python3.6): pyspark.tests.test_taskcontext
Finished test(python3.6): pyspark.tests.test_serializers (11s)
Starting test(python3.6): pyspark.tests.test_util
Finished test(python3.6): pyspark.tests.test_shuffle (11s)
Starting test(python3.6): pyspark.tests.test_worker
Finished test(python3.6): pyspark.tests.test_util (7s)
Starting test(python3.6): pyspark.accumulators
Finished test(python3.6): pyspark.accumulators (7s)
Starting test(python3.6): pyspark.broadcast
Finished test(python3.6): pyspark.broadcast (7s)
Starting test(python3.6): pyspark.conf
Finished test(python3.6): pyspark.tests.test_worker (19s)
Starting test(python3.6): pyspark.context
Finished test(python3.6): pyspark.conf (5s)
Starting test(python3.6): pyspark.ml.classification
Finished test(python3.6): pyspark.tests.test_taskcontext (43s)
Starting test(python3.6): pyspark.ml.clustering
Finished test(python3.6): pyspark.context (24s)
Starting test(python3.6): pyspark.ml.evaluation
Finished test(python3.6): pyspark.tests.test_rdd (101s)
Starting test(python3.6): pyspark.ml.feature
Finished test(python3.6): pyspark.ml.evaluation (30s)
Starting test(python3.6): pyspark.ml.fpm
Finished test(python3.6): pyspark.ml.clustering (51s)
Starting test(python3.6): pyspark.ml.functions
Finished test(python3.6): pyspark.ml.fpm (26s)
Starting test(python3.6): pyspark.ml.image
Finished test(python3.6): pyspark.ml.functions (18s)
Starting test(python3.6): pyspark.ml.linalg.__init__
Finished test(python3.6): pyspark.ml.linalg.__init__ (0s)
Starting test(python3.6): pyspark.ml.recommendation
Finished test(python3.6): pyspark.ml.image (7s)
Starting test(python3.6): pyspark.ml.regression
Finished test(python3.6): pyspark.ml.classification (89s)
Starting test(python3.6): pyspark.ml.stat
Finished test(python3.6): pyspark.ml.stat (29s)
Starting test(python3.6): pyspark.ml.tuning
Finished test(python3.6): pyspark.ml.feature (109s)
Starting test(python3.6): pyspark.mllib.classification
Finished test(python3.6): pyspark.ml.regression (54s)
Starting test(python3.6): pyspark.mllib.clustering
Finished test(python3.6): pyspark.ml.tuning (39s)
Starting test(python3.6): pyspark.mllib.evaluation
Finished test(python3.6): pyspark.mllib.evaluation (27s)
Starting test(python3.6): pyspark.mllib.feature
Finished test(python3.6): pyspark.mllib.classification (47s)
Starting test(python3.6): pyspark.mllib.fpm
Finished test(python3.6): pyspark.ml.recommendation (116s)
Starting test(python3.6): pyspark.mllib.linalg.__init__
Finished test(python3.6): pyspark.mllib.linalg.__init__ (0s)
Starting test(python3.6): pyspark.mllib.linalg.distributed
Finished test(python3.6): pyspark.mllib.clustering (62s)
Starting test(python3.6): pyspark.mllib.random
Finished test(python3.6): pyspark.mllib.fpm (24s)
Starting test(python3.6): pyspark.mllib.recommendation
Finished test(python3.6): pyspark.mllib.feature (29s)
Starting test(python3.6): pyspark.mllib.regression
Finished test(python3.6): pyspark.mllib.random (13s)
Starting test(python3.6): pyspark.mllib.stat.KernelDensity
Finished test(python3.6): pyspark.mllib.stat.KernelDensity (0s)
Starting test(python3.6): pyspark.mllib.stat._statistics
Finished test(python3.6): pyspark.mllib.stat._statistics (23s)
Starting test(python3.6): pyspark.mllib.tree
Finished test(python3.6): pyspark.mllib.regression (35s)
Starting test(python3.6): pyspark.mllib.util
Finished test(python3.6): pyspark.mllib.recommendation (47s)
Starting test(python3.6): pyspark.profiler
Finished test(python3.6): pyspark.mllib.tree (23s)
Starting test(python3.6): pyspark.rdd
Finished test(python3.6): pyspark.mllib.linalg.distributed (64s)
Starting test(python3.6): pyspark.serializers
Finished test(python3.6): pyspark.profiler (9s)
Starting test(python3.6): pyspark.shuffle
Finished test(python3.6): pyspark.shuffle (0s)
Starting test(python3.6): pyspark.sql.avro.functions
Finished test(python3.6): pyspark.mllib.util (21s)
Starting test(python3.6): pyspark.sql.catalog
Finished test(python3.6): pyspark.serializers (11s)
Starting test(python3.6): pyspark.sql.column
Finished test(python3.6): pyspark.sql.avro.functions (17s)
Starting test(python3.6): pyspark.sql.conf
Finished test(python3.6): pyspark.sql.catalog (16s)
Starting test(python3.6): pyspark.sql.context
Finished test(python3.6): pyspark.sql.conf (7s)
Starting test(python3.6): pyspark.sql.dataframe
Finished test(python3.6): pyspark.rdd (42s)
Starting test(python3.6): pyspark.sql.functions
Finished test(python3.6): pyspark.sql.column (33s)
Starting test(python3.6): pyspark.sql.group
Finished test(python3.6): pyspark.sql.context (24s)
Starting test(python3.6): pyspark.sql.pandas.conversion
Finished test(python3.6): pyspark.sql.pandas.conversion (8s)
Starting test(python3.6): pyspark.sql.pandas.functions
Finished test(python3.6): pyspark.sql.pandas.functions (15s)
Starting test(python3.6): pyspark.sql.pandas.group_ops
Finished test(python3.6): pyspark.sql.group (37s)
Starting test(python3.6): pyspark.sql.pandas.map_ops
Finished test(python3.6): pyspark.sql.pandas.group_ops (16s)
Starting test(python3.6): pyspark.sql.pandas.serializers
Finished test(python3.6): pyspark.sql.pandas.serializers (0s)
Starting test(python3.6): pyspark.sql.pandas.typehints
Finished test(python3.6): pyspark.sql.pandas.typehints (0s)
Starting test(python3.6): pyspark.sql.pandas.types
Finished test(python3.6): pyspark.sql.pandas.types (0s)
Starting test(python3.6): pyspark.sql.pandas.utils
Finished test(python3.6): pyspark.sql.pandas.utils (0s)
Starting test(python3.6): pyspark.sql.readwriter
Finished test(python3.6): pyspark.sql.pandas.map_ops (12s)
Starting test(python3.6): pyspark.sql.session
Finished test(python3.6): pyspark.sql.dataframe (76s)
Starting test(python3.6): pyspark.sql.streaming
Finished test(python3.6): pyspark.sql.functions (71s)
Starting test(python3.6): pyspark.sql.types
Finished test(python3.6): pyspark.sql.readwriter (25s)
Starting test(python3.6): pyspark.sql.udf
Finished test(python3.6): pyspark.sql.session (23s)
Starting test(python3.6): pyspark.sql.window
Finished test(python3.6): pyspark.sql.streaming (16s)
Starting test(python3.6): pyspark.streaming.util
Finished test(python3.6): pyspark.streaming.util (0s)
Starting test(python3.6): pyspark.util
Finished test(python3.6): pyspark.util (0s)
Finished test(python3.6): pyspark.sql.types (8s)
Finished test(python3.6): pyspark.sql.udf (18s)
Finished test(python3.6): pyspark.sql.window (16s)
Tests passed in 919 seconds

Skipped tests in pyspark.sql.tests.test_arrow with python3.6:
    test_createDataFrame_column_name_encoding (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_does_not_modify_input (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_fallback_disabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_fallback_enabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_respect_session_timezone (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_toggle (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_array_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_incorrect_schema (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_int_col_names (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_names (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_schema (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_single_data_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_filtered_frame (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_no_partition_frame (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_null_conversion (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_pandas_round_trip (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_propagates_spark_exception (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_schema_conversion_roundtrip (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_timestamp_dst (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_timestamp_nat (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_arrow_toggle (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_batch_order (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_fallback_disabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_fallback_enabled (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_respect_session_timezone (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_with_array_type (pyspark.sql.tests.test_arrow.ArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_column_name_encoding (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_does_not_modify_input (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_fallback_disabled (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_fallback_enabled (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_respect_session_timezone (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_toggle (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_array_type (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_incorrect_schema (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_int_col_names (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_names (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_schema (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_createDataFrame_with_single_data_type (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_filtered_frame (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_no_partition_frame (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_null_conversion (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_pandas_round_trip (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_propagates_spark_exception (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_schema_conversion_roundtrip (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_timestamp_dst (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_timestamp_nat (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_arrow_toggle (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_batch_order (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_fallback_disabled (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_fallback_enabled (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_respect_session_timezone (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_toPandas_with_array_type (pyspark.sql.tests.test_arrow.EncryptionArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_exception_by_max_results (pyspark.sql.tests.test_arrow.MaxResultArrowTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_dataframe with python3.6:
    test_create_dataframe_required_pandas_not_found (pyspark.sql.tests.test_dataframe.DataFrameTests) ... skipped 'Required Pandas was found.'
    test_to_pandas_required_pandas_not_found (pyspark.sql.tests.test_dataframe.DataFrameTests) ... skipped 'Required Pandas was found.'
    test_query_execution_listener_on_collect_with_arrow (pyspark.sql.tests.test_dataframe.QueryExecutionListenerTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_cogrouped_map with python3.6:
    test_complex_group_by (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_different_schemas (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_empty_group_by (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_left_group_empty (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_scalar_udfs_followed_by_cogrouby_apply (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_right_group_empty (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_simple (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_with_key_complex (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_with_key_left (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_with_key_left_group_empty (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_with_key_right (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_with_key_right_group_empty (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_wrong_args (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_wrong_return_type (pyspark.sql.tests.test_pandas_cogrouped_map.CogroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_grouped_map with python3.6:
    test_array_type_correct (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_coerce (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_column_order (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_complex_groupby (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_datatype_string (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_decorator (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_empty_groupby (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_grouped_over_window (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_grouped_over_window_with_key (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_grouped_with_empty_partition (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_scalar_udfs_followed_by_grouby_apply (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_positional_assignment_conf (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_register_grouped_map_udf (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_self_join_with_pandas (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_supported_types (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_timestamp_dst (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_udf_with_key (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_unsupported_types (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_wrong_args (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_wrong_return_type (pyspark.sql.tests.test_pandas_grouped_map.GroupedMapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_map with python3.6:
    test_chain_map_partitions_in_pandas (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_different_output_length (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_empty_iterator (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_empty_rows (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_map_partitions_in_pandas (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_multiple_columns (pyspark.sql.tests.test_pandas_map.MapInPandasTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_udf with python3.6:
    test_pandas_udf_arrow_overflow (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_pandas_udf_basic (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_pandas_udf_decorator (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_pandas_udf_detect_unsafe_type_conversion (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_stopiteration_in_udf (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_udf_wrong_arg (pyspark.sql.tests.test_pandas_udf.PandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_udf_grouped_agg with python3.6:
    test_alias (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_array_type (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_basic (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_complex_expressions (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_complex_groupby (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_grouped_with_empty_partition (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_grouped_without_group_by_clause (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_invalid_args (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_manual (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_sql (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_udfs (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_multiple_udfs (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_register_vectorized_udf_basic (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_retain_group_columns (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_unsupported_types (pyspark.sql.tests.test_pandas_udf_grouped_agg.GroupedAggPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_udf_scalar with python3.6:
    test_datasource_with_udf (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_udf (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_udf_and_sql (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_nondeterministic_vectorized_udf (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_nondeterministic_vectorized_udf_in_aggregate (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_pandas_udf_nested_arrays (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_pandas_udf_tokenize (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_register_nondeterministic_vectorized_udf_basic (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_register_vectorized_udf_basic (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_scalar_iter_udf_close (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_scalar_iter_udf_close_early (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_scalar_iter_udf_init (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_timestamp_dst (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_type_annotation (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_array_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_basic (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_chained (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_chained_struct_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_check_config (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_complex (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_datatype_string (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_dates (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_decorator (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_empty_partition (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_exception (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_invalid_length (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_nested_struct (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_array (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_binary (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_boolean (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_byte (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_decimal (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_double (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_float (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_int (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_long (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_short (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_null_string (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_return_scalar (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_return_timestamp_tz (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_string_in_udf (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_struct_complex (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_struct_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_struct_with_empty_partition (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_timestamps (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_timestamps_respect_session_timezone (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_unsupported_types (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_varargs (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_vectorized_udf_wrong_return_type (pyspark.sql.tests.test_pandas_udf_scalar.ScalarPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_udf_typehints with python3.6:
    test_group_agg_udf_type_hint (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_scalar_iter_udf_type_hint (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_scalar_udf_type_hint (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_type_annotation_group_agg (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_type_annotation_negative (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_type_annotation_scalar (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_type_annotation_scalar_iter (pyspark.sql.tests.test_pandas_udf_typehints.PandasUDFTypeHintsTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.sql.tests.test_pandas_udf_window with python3.6:
    test_array_type (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_bounded_mixed (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_bounded_simple (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_growing_window (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_invalid_args (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_sql (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_sql_and_udf (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_mixed_udf (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_multiple_udfs (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_replace_existing (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_shrinking_window (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_simple (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_sliding_window (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'
    test_without_partitionBy (pyspark.sql.tests.test_pandas_udf_window.WindowPandasUDFTests) ... skipped 'PyArrow >= 0.15.1 must be installed; however, it was not found.'

Skipped tests in pyspark.streaming.tests.test_kinesis with python3.6:
    test_kinesis_stream (pyspark.streaming.tests.test_kinesis.KinesisStreamTests) ... skipped "Skipping all Kinesis Python tests as environmental variable 'ENABLE_KINESIS_TESTS' was not set."
    test_kinesis_stream_api (pyspark.streaming.tests.test_kinesis.KinesisStreamTests) ... skipped "Skipping all Kinesis Python tests as environmental variable 'ENABLE_KINESIS_TESTS' was not set."

Skipped tests in pyspark.tests.test_readwrite with python3.6:
    test_sequencefiles (pyspark.tests.test_readwrite.InputFormatTests) ... skipped 'serialize array of byte'
    test_newhadoop_with_array (pyspark.tests.test_readwrite.OutputFormatTests) ... skipped 'serialize of array'
    test_sequencefiles (pyspark.tests.test_readwrite.OutputFormatTests) ... skipped 'serialize array of byte'
Archiving artifacts
Finished: SUCCESS