FailedConsole Output

GitHub pull request #5143 of commit 06c445fa4deed19491bdcf2feb9b436e5c2ca732 automatically merged.
[EnvInject] - Loading node environment variables.
Building remotely on ubuntu-testing (ubuntu-testing-gpu) in workspace /home/jenkins/workspace/Ray-Perf-Integration-PRB
[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Done
Cloning the remote Git repository
Cloning repository https://github.com/ray-project/ray.git
 > git init /home/jenkins/workspace/Ray-Perf-Integration-PRB # timeout=10
Fetching upstream changes from https://github.com/ray-project/ray.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/ray-project/ray.git +refs/heads/*:refs/remotes/origin/*
 > git config remote.origin.url https://github.com/ray-project/ray.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/ray-project/ray.git # timeout=10
Fetching upstream changes from https://github.com/ray-project/ray.git
 > git fetch --tags --progress https://github.com/ray-project/ray.git +refs/pull/*:refs/remotes/origin/pr/*
 > git rev-parse origin/pr/5143/merge^{commit} # timeout=10
 > git branch -a -v --no-abbrev --contains ac4bce8a9b9699e58b93e988e641254036c9bf4e # timeout=10
Checking out Revision ac4bce8a9b9699e58b93e988e641254036c9bf4e (origin/pr/5143/merge)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f ac4bce8a9b9699e58b93e988e641254036c9bf4e
 > git rev-list f59308a9117f00c45fa2c7dd10c9349abc00187b # timeout=10
First time build. Skipping changelog.
[Ray-Perf-Integration-PRB] $ /bin/bash /tmp/hudson6728044425974215826.sh
+ [[ 2 -gt 0 ]]
+ key=--output-sha
+ case $key in
+ OUTPUT_SHA=YES
+ shift
+ [[ 1 -gt 0 ]]
+ key=--no-cache
+ case $key in
+ NO_CACHE=--no-cache
+ shift
+ [[ 0 -gt 0 ]]
+ '[' YES ']'
++ docker build --no-cache -q -t ray-project/base-deps docker/base-deps
+ IMAGE_SHA=sha256:eac19b63fad6f872782df257f48739a33c2207ae0496e22b178346c6c1d5bf64
+ git rev-parse HEAD
++ git rev-parse HEAD
+ git archive -o ./docker/deploy/ray.tar ac4bce8a9b9699e58b93e988e641254036c9bf4e
+ '[' YES ']'
++ docker build --no-cache -q -t ray-project/deploy docker/deploy
+ IMAGE_SHA=sha256:5c0583e73cf676547c07e24bbdcdc69117f504f981b699264cb366b0bb849c2a
+ rm ./docker/deploy/ray.tar ./docker/deploy/git-rev
+ '[' '!' ']'
+ '[' YES ']'
++ docker build --no-cache -q -t ray-project/examples docker/examples
+ IMAGE_SHA=sha256:896df471550e50595f68fcee102cd72e514c083f31a02b45086773bf00d597b1
+ '[' YES ']'
+ echo sha256:896df471550e50595f68fcee102cd72e514c083f31a02b45086773bf00d597b1
+ sed s/sha256://
Using Docker image 896df471550e50595f68fcee102cd72e514c083f31a02b45086773bf00d597b1
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
+++ dirname ./ci/jenkins_tests/perf_integration_tests/run_perf_integration.sh
++ cd ./ci/jenkins_tests/perf_integration_tests
++ pwd
+ ROOT_DIR=/ray/ci/jenkins_tests/perf_integration_tests
+ pushd /ray/ci/jenkins_tests/perf_integration_tests
+ python -m pip install pytest-benchmark
/ray/ci/jenkins_tests/perf_integration_tests /ray
Collecting pytest-benchmark
  Downloading https://files.pythonhosted.org/packages/ab/0f/8488bebc18924bbf5ed9eff3d3cecd0287446c88962f10e99fa727a3d2d7/pytest_benchmark-3.2.2-py2.py3-none-any.whl (48kB)
Collecting py-cpuinfo (from pytest-benchmark)
  Downloading https://files.pythonhosted.org/packages/42/60/63f28a5401da733043abe7053e7d9591491b4784c4f87c339bf51215aa0a/py-cpuinfo-5.0.0.tar.gz (82kB)
Requirement already satisfied: pytest>=3.8 in /opt/conda/lib/python3.6/site-packages (from pytest-benchmark) (5.0.1)
Requirement already satisfied: importlib-metadata>=0.12 in /opt/conda/lib/python3.6/site-packages (from pytest>=3.8->pytest-benchmark) (0.18)
Requirement already satisfied: attrs>=17.4.0 in /opt/conda/lib/python3.6/site-packages (from pytest>=3.8->pytest-benchmark) (18.1.0)
Requirement already satisfied: py>=1.5.0 in /opt/conda/lib/python3.6/site-packages (from pytest>=3.8->pytest-benchmark) (1.5.3)
Requirement already satisfied: wcwidth in /opt/conda/lib/python3.6/site-packages (from pytest>=3.8->pytest-benchmark) (0.1.7)
Requirement already satisfied: more-itertools>=4.0.0 in /opt/conda/lib/python3.6/site-packages (from pytest>=3.8->pytest-benchmark) (4.1.0)
Requirement already satisfied: atomicwrites>=1.0 in /opt/conda/lib/python3.6/site-packages (from pytest>=3.8->pytest-benchmark) (1.3.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in /opt/conda/lib/python3.6/site-packages (from pytest>=3.8->pytest-benchmark) (0.12.0)
Requirement already satisfied: packaging in /opt/conda/lib/python3.6/site-packages (from pytest>=3.8->pytest-benchmark) (17.1)
Requirement already satisfied: zipp>=0.5 in /opt/conda/lib/python3.6/site-packages (from importlib-metadata>=0.12->pytest>=3.8->pytest-benchmark) (0.5.2)
Requirement already satisfied: six<2.0.0,>=1.0.0 in /opt/conda/lib/python3.6/site-packages (from more-itertools>=4.0.0->pytest>=3.8->pytest-benchmark) (1.11.0)
Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/lib/python3.6/site-packages (from packaging->pytest>=3.8->pytest-benchmark) (2.2.0)
Building wheels for collected packages: py-cpuinfo
  Running setup.py bdist_wheel for py-cpuinfo: started
  Running setup.py bdist_wheel for py-cpuinfo: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/01/7e/a9/b982d0fea22b7e4ae5619de949570cde5ad55420cec16e86a5
Successfully built py-cpuinfo
distributed 1.21.8 requires dask>=0.17.0, which is not installed.
distributed 1.21.8 requires msgpack, which is not installed.
blaze 0.11.3 requires dask[dataframe], which is not installed.
jupyterlab-server 1.0.0 has requirement jsonschema>=3.0.1, but you'll have jsonschema 2.6.0 which is incompatible.
Installing collected packages: py-cpuinfo, pytest-benchmark
Successfully installed py-cpuinfo-5.0.0 pytest-benchmark-3.2.2
You are using pip version 10.0.1, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
+ pip install -U https://ray-wheels.s3-us-west-2.amazonaws.com/latest/ray-0.8.0.dev2-cp36-cp36m-manylinux1_x86_64.whl
Collecting ray==0.8.0.dev2 from https://ray-wheels.s3-us-west-2.amazonaws.com/latest/ray-0.8.0.dev2-cp36-cp36m-manylinux1_x86_64.whl
  Downloading https://ray-wheels.s3-us-west-2.amazonaws.com/latest/ray-0.8.0.dev2-cp36-cp36m-manylinux1_x86_64.whl (48.4MB)
Requirement not upgraded as not directly required: pyyaml in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (3.12)
Requirement not upgraded as not directly required: colorama in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (0.3.9)
Requirement not upgraded as not directly required: click in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (6.7)
Requirement not upgraded as not directly required: filelock in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (3.0.4)
Requirement not upgraded as not directly required: funcsigs in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (1.0.2)
Requirement not upgraded as not directly required: redis in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (3.2.1)
Requirement not upgraded as not directly required: pytest in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (5.0.1)
Requirement not upgraded as not directly required: protobuf>=3.8.0 in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (3.9.0)
Requirement not upgraded as not directly required: six>=1.0.0 in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (1.11.0)
Requirement not upgraded as not directly required: numpy>=1.14 in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (1.16.4)
Requirement not upgraded as not directly required: py>=1.5.0 in /opt/conda/lib/python3.6/site-packages (from pytest->ray==0.8.0.dev2) (1.5.3)
Requirement not upgraded as not directly required: more-itertools>=4.0.0 in /opt/conda/lib/python3.6/site-packages (from pytest->ray==0.8.0.dev2) (4.1.0)
Requirement not upgraded as not directly required: atomicwrites>=1.0 in /opt/conda/lib/python3.6/site-packages (from pytest->ray==0.8.0.dev2) (1.3.0)
Requirement not upgraded as not directly required: pluggy<1.0,>=0.12 in /opt/conda/lib/python3.6/site-packages (from pytest->ray==0.8.0.dev2) (0.12.0)
Requirement not upgraded as not directly required: attrs>=17.4.0 in /opt/conda/lib/python3.6/site-packages (from pytest->ray==0.8.0.dev2) (18.1.0)
Requirement not upgraded as not directly required: wcwidth in /opt/conda/lib/python3.6/site-packages (from pytest->ray==0.8.0.dev2) (0.1.7)
Requirement not upgraded as not directly required: packaging in /opt/conda/lib/python3.6/site-packages (from pytest->ray==0.8.0.dev2) (17.1)
Requirement not upgraded as not directly required: importlib-metadata>=0.12 in /opt/conda/lib/python3.6/site-packages (from pytest->ray==0.8.0.dev2) (0.18)
Requirement not upgraded as not directly required: setuptools in /opt/conda/lib/python3.6/site-packages (from protobuf>=3.8.0->ray==0.8.0.dev2) (41.0.1)
Requirement not upgraded as not directly required: pyparsing>=2.0.2 in /opt/conda/lib/python3.6/site-packages (from packaging->pytest->ray==0.8.0.dev2) (2.2.0)
Requirement not upgraded as not directly required: zipp>=0.5 in /opt/conda/lib/python3.6/site-packages (from importlib-metadata>=0.12->pytest->ray==0.8.0.dev2) (0.5.2)
distributed 1.21.8 requires dask>=0.17.0, which is not installed.
distributed 1.21.8 requires msgpack, which is not installed.
blaze 0.11.3 requires dask[dataframe], which is not installed.
jupyterlab-server 1.0.0 has requirement jsonschema>=3.0.1, but you'll have jsonschema 2.6.0 which is incompatible.
Installing collected packages: ray
  Found existing installation: ray 0.8.0.dev2
    Uninstalling ray-0.8.0.dev2:
      Successfully uninstalled ray-0.8.0.dev2
Successfully installed ray-0.8.0.dev2
You are using pip version 10.0.1, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
+ python -m pytest --benchmark-autosave --benchmark-min-rounds=10 '--benchmark-columns=min, max, mean' /ray/ci/jenkins_tests/perf_integration_tests/../../../python/ray/tests/perf_integration_tests/test_perf_integration.py
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-5.0.1, py-1.5.3, pluggy-0.12.0
benchmark: 3.2.2 (defaults: timer=time.perf_counter disable_gc=False min_rounds=10 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /ray/python
plugins: arraydiff-0.2, openfiles-0.3.0, doctestplus-0.1.3, benchmark-3.2.2, remotedata-0.3.1, timeout-1.3.3
collected 12 items

../../../python/ray/tests/perf_integration_tests/test_perf_integration.py . [  8%]
...........                                                              [100%]
Saved benchmark data in: /ray/ci/jenkins_tests/perf_integration_tests/.benchmarks/Linux-CPython-3.6-64bit/0001_unversioned_20190713_225612.json




Computing stats ...
Computing stats ... group 1/1
Computing stats ... group 1/1: min
Computing stats ... group 1/1: min (1/12)
Computing stats ... group 1/1: min (2/12)
Computing stats ... group 1/1: min (3/12)
Computing stats ... group 1/1: min (4/12)
Computing stats ... group 1/1: min (5/12)
Computing stats ... group 1/1: min (6/12)
Computing stats ... group 1/1: min (7/12)
Computing stats ... group 1/1: min (8/12)
Computing stats ... group 1/1: min (9/12)
Computing stats ... group 1/1: min (10/12)
Computing stats ... group 1/1: min (11/12)
Computing stats ... group 1/1: min (12/12)
Computing stats ... group 1/1: min (1/12)
Computing stats ... group 1/1: min (2/12)
Computing stats ... group 1/1: min (3/12)
Computing stats ... group 1/1: min (4/12)
Computing stats ... group 1/1: min (5/12)
Computing stats ... group 1/1: min (6/12)
Computing stats ... group 1/1: min (7/12)
Computing stats ... group 1/1: min (8/12)
Computing stats ... group 1/1: min (9/12)
Computing stats ... group 1/1: min (10/12)
Computing stats ... group 1/1: min (11/12)
Computing stats ... group 1/1: min (12/12)
Computing stats ... group 1/1: max
Computing stats ... group 1/1: max (1/12)
Computing stats ... group 1/1: max (2/12)
Computing stats ... group 1/1: max (3/12)
Computing stats ... group 1/1: max (4/12)
Computing stats ... group 1/1: max (5/12)
Computing stats ... group 1/1: max (6/12)
Computing stats ... group 1/1: max (7/12)
Computing stats ... group 1/1: max (8/12)
Computing stats ... group 1/1: max (9/12)
Computing stats ... group 1/1: max (10/12)
Computing stats ... group 1/1: max (11/12)
Computing stats ... group 1/1: max (12/12)
Computing stats ... group 1/1: max (1/12)
Computing stats ... group 1/1: max (2/12)
Computing stats ... group 1/1: max (3/12)
Computing stats ... group 1/1: max (4/12)
Computing stats ... group 1/1: max (5/12)
Computing stats ... group 1/1: max (6/12)
Computing stats ... group 1/1: max (7/12)
Computing stats ... group 1/1: max (8/12)
Computing stats ... group 1/1: max (9/12)
Computing stats ... group 1/1: max (10/12)
Computing stats ... group 1/1: max (11/12)
Computing stats ... group 1/1: max (12/12)
Computing stats ... group 1/1: mean
Computing stats ... group 1/1: mean (1/12)
Computing stats ... group 1/1: mean (2/12)
Computing stats ... group 1/1: mean (3/12)
Computing stats ... group 1/1: mean (4/12)
Computing stats ... group 1/1: mean (5/12)
Computing stats ... group 1/1: mean (6/12)
Computing stats ... group 1/1: mean (7/12)
Computing stats ... group 1/1: mean (8/12)
Computing stats ... group 1/1: mean (9/12)
Computing stats ... group 1/1: mean (10/12)
Computing stats ... group 1/1: mean (11/12)
Computing stats ... group 1/1: mean (12/12)
Computing stats ... group 1/1: mean (1/12)
Computing stats ... group 1/1: mean (2/12)
Computing stats ... group 1/1: mean (3/12)
Computing stats ... group 1/1: mean (4/12)
Computing stats ... group 1/1: mean (5/12)
Computing stats ... group 1/1: mean (6/12)
Computing stats ... group 1/1: mean (7/12)
Computing stats ... group 1/1: mean (8/12)
Computing stats ... group 1/1: mean (9/12)
Computing stats ... group 1/1: mean (10/12)
Computing stats ... group 1/1: mean (11/12)
Computing stats ... group 1/1: mean (12/12)
Computing stats ... group 1/1: median
Computing stats ... group 1/1: median (1/12)
Computing stats ... group 1/1: median (2/12)
Computing stats ... group 1/1: median (3/12)
Computing stats ... group 1/1: median (4/12)
Computing stats ... group 1/1: median (5/12)
Computing stats ... group 1/1: median (6/12)
Computing stats ... group 1/1: median (7/12)
Computing stats ... group 1/1: median (8/12)
Computing stats ... group 1/1: median (9/12)
Computing stats ... group 1/1: median (10/12)
Computing stats ... group 1/1: median (11/12)
Computing stats ... group 1/1: median (12/12)
Computing stats ... group 1/1: median (1/12)
Computing stats ... group 1/1: median (2/12)
Computing stats ... group 1/1: median (3/12)
Computing stats ... group 1/1: median (4/12)
Computing stats ... group 1/1: median (5/12)
Computing stats ... group 1/1: median (6/12)
Computing stats ... group 1/1: median (7/12)
Computing stats ... group 1/1: median (8/12)
Computing stats ... group 1/1: median (9/12)
Computing stats ... group 1/1: median (10/12)
Computing stats ... group 1/1: median (11/12)
Computing stats ... group 1/1: median (12/12)
Computing stats ... group 1/1: iqr
Computing stats ... group 1/1: iqr (1/12)
Computing stats ... group 1/1: iqr (2/12)
Computing stats ... group 1/1: iqr (3/12)
Computing stats ... group 1/1: iqr (4/12)
Computing stats ... group 1/1: iqr (5/12)
Computing stats ... group 1/1: iqr (6/12)
Computing stats ... group 1/1: iqr (7/12)
Computing stats ... group 1/1: iqr (8/12)
Computing stats ... group 1/1: iqr (9/12)
Computing stats ... group 1/1: iqr (10/12)
Computing stats ... group 1/1: iqr (11/12)
Computing stats ... group 1/1: iqr (12/12)
Computing stats ... group 1/1: iqr (1/12)
Computing stats ... group 1/1: iqr (2/12)
Computing stats ... group 1/1: iqr (3/12)
Computing stats ... group 1/1: iqr (4/12)
Computing stats ... group 1/1: iqr (5/12)
Computing stats ... group 1/1: iqr (6/12)
Computing stats ... group 1/1: iqr (7/12)
Computing stats ... group 1/1: iqr (8/12)
Computing stats ... group 1/1: iqr (9/12)
Computing stats ... group 1/1: iqr (10/12)
Computing stats ... group 1/1: iqr (11/12)
Computing stats ... group 1/1: iqr (12/12)
Computing stats ... group 1/1: stddev
Computing stats ... group 1/1: stddev (1/12)
Computing stats ... group 1/1: stddev (2/12)
Computing stats ... group 1/1: stddev (3/12)
Computing stats ... group 1/1: stddev (4/12)
Computing stats ... group 1/1: stddev (5/12)
Computing stats ... group 1/1: stddev (6/12)
Computing stats ... group 1/1: stddev (7/12)
Computing stats ... group 1/1: stddev (8/12)
Computing stats ... group 1/1: stddev (9/12)
Computing stats ... group 1/1: stddev (10/12)
Computing stats ... group 1/1: stddev (11/12)
Computing stats ... group 1/1: stddev (12/12)
Computing stats ... group 1/1: stddev (1/12)
Computing stats ... group 1/1: stddev (2/12)
Computing stats ... group 1/1: stddev (3/12)
Computing stats ... group 1/1: stddev (4/12)
Computing stats ... group 1/1: stddev (5/12)
Computing stats ... group 1/1: stddev (6/12)
Computing stats ... group 1/1: stddev (7/12)
Computing stats ... group 1/1: stddev (8/12)
Computing stats ... group 1/1: stddev (9/12)
Computing stats ... group 1/1: stddev (10/12)
Computing stats ... group 1/1: stddev (11/12)
Computing stats ... group 1/1: stddev (12/12)
Computing stats ... group 1/1: ops
Computing stats ... group 1/1: ops (1/12)
Computing stats ... group 1/1: ops (2/12)
Computing stats ... group 1/1: ops (3/12)
Computing stats ... group 1/1: ops (4/12)
Computing stats ... group 1/1: ops (5/12)
Computing stats ... group 1/1: ops (6/12)
Computing stats ... group 1/1: ops (7/12)
Computing stats ... group 1/1: ops (8/12)
Computing stats ... group 1/1: ops (9/12)
Computing stats ... group 1/1: ops (10/12)
Computing stats ... group 1/1: ops (11/12)
Computing stats ... group 1/1: ops (12/12)
Computing stats ... group 1/1: ops (1/12)
Computing stats ... group 1/1: ops (2/12)
Computing stats ... group 1/1: ops (3/12)
Computing stats ... group 1/1: ops (4/12)
Computing stats ... group 1/1: ops (5/12)
Computing stats ... group 1/1: ops (6/12)
Computing stats ... group 1/1: ops (7/12)
Computing stats ... group 1/1: ops (8/12)
Computing stats ... group 1/1: ops (9/12)
Computing stats ... group 1/1: ops (10/12)
Computing stats ... group 1/1: ops (11/12)
Computing stats ... group 1/1: ops (12/12)
Computing stats ... group 1/1: ops: outliers
Computing stats ... group 1/1: ops: outliers (1/12)
Computing stats ... group 1/1: ops: outliers (2/12)
Computing stats ... group 1/1: ops: outliers (3/12)
Computing stats ... group 1/1: ops: outliers (4/12)
Computing stats ... group 1/1: ops: outliers (5/12)
Computing stats ... group 1/1: ops: outliers (6/12)
Computing stats ... group 1/1: ops: outliers (7/12)
Computing stats ... group 1/1: ops: outliers (8/12)
Computing stats ... group 1/1: ops: outliers (9/12)
Computing stats ... group 1/1: ops: outliers (10/12)
Computing stats ... group 1/1: ops: outliers (11/12)
Computing stats ... group 1/1: ops: outliers (12/12)
Computing stats ... group 1/1: ops: rounds
Computing stats ... group 1/1: ops: rounds (1/12)
Computing stats ... group 1/1: ops: rounds (2/12)
Computing stats ... group 1/1: ops: rounds (3/12)
Computing stats ... group 1/1: ops: rounds (4/12)
Computing stats ... group 1/1: ops: rounds (5/12)
Computing stats ... group 1/1: ops: rounds (6/12)
Computing stats ... group 1/1: ops: rounds (7/12)
Computing stats ... group 1/1: ops: rounds (8/12)
Computing stats ... group 1/1: ops: rounds (9/12)
Computing stats ... group 1/1: ops: rounds (10/12)
Computing stats ... group 1/1: ops: rounds (11/12)
Computing stats ... group 1/1: ops: rounds (12/12)
Computing stats ... group 1/1: ops: iterations
Computing stats ... group 1/1: ops: iterations (1/12)
Computing stats ... group 1/1: ops: iterations (2/12)
Computing stats ... group 1/1: ops: iterations (3/12)
Computing stats ... group 1/1: ops: iterations (4/12)
Computing stats ... group 1/1: ops: iterations (5/12)
Computing stats ... group 1/1: ops: iterations (6/12)
Computing stats ... group 1/1: ops: iterations (7/12)
Computing stats ... group 1/1: ops: iterations (8/12)
Computing stats ... group 1/1: ops: iterations (9/12)
Computing stats ... group 1/1: ops: iterations (10/12)
Computing stats ... group 1/1: ops: iterations (11/12)
Computing stats ... group 1/1: ops: iterations (12/12)
--------------------------------------------- benchmark: 12 tests ----------------------------------------------
Name (time in ms)                                  Min                     Max                    Mean          
----------------------------------------------------------------------------------------------------------------
test_transfer_performance[1000-500]            78.0780 (1.0)           98.9794 (1.0)           88.3041 (1.0)    
test_transfer_performance[1000-5000]          103.9944 (1.33)         113.6821 (1.15)         108.2724 (1.23)   
test_task_forward[1000_tasks]                 320.1597 (4.10)         377.1385 (3.81)         351.0253 (3.98)   
test_transfer_performance[10000-500]          705.7309 (9.04)         803.4605 (8.12)         760.8389 (8.62)   
test_transfer_performance[10000-5000]         722.6228 (9.26)         795.0024 (8.03)         765.3734 (8.67)   
test_task_forward[10000_tasks]              2,942.9156 (37.69)      3,326.9708 (33.61)      3,073.8600 (34.81)  
test_task_submission[100_tasks]            22,535.5087 (288.63)    22,826.6716 (230.62)    22,624.1203 (256.21) 
test_task_submission[1000_tasks]           23,234.5893 (297.58)    24,211.8219 (244.61)    23,567.0319 (266.88) 
test_task_submission[10000_tasks]          23,285.6245 (298.24)    23,874.6958 (241.21)    23,551.8852 (266.71) 
test_task_submission[10_tasks]             29,259.1614 (374.74)    30,245.1633 (305.57)    29,756.6330 (336.98) 
test_task_submission[1_tasks]             112,663.9791 (>1000.0)  135,063.5094 (>1000.0)  129,188.0302 (>1000.0)
test_task_submission[100000_tasks]        149,491.9673 (>1000.0)  201,440.6912 (>1000.0)  161,452.0020 (>1000.0)
----------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean
========================= 12 passed in 4761.13 seconds =========================
+ pushd /ray/ci/jenkins_tests/perf_integration_tests/../../../python
+ python -m pip install -e .
/ray/python /ray/ci/jenkins_tests/perf_integration_tests /ray
Obtaining file:///ray/python
Requirement already satisfied: numpy>=1.14 in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (1.16.4)
Requirement already satisfied: filelock in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (3.0.4)
Requirement already satisfied: funcsigs in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (1.0.2)
Requirement already satisfied: click in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (6.7)
Requirement already satisfied: colorama in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (0.3.9)
Requirement already satisfied: pytest in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (5.0.1)
Requirement already satisfied: pyyaml in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (3.12)
Requirement already satisfied: redis in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (3.2.1)
Requirement already satisfied: six>=1.0.0 in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (1.11.0)
Requirement already satisfied: protobuf>=3.8.0 in /opt/conda/lib/python3.6/site-packages (from ray==0.8.0.dev2) (3.9.0)
Requirement already satisfied: importlib-metadata>=0.12 in /opt/conda/lib/python3.6/site-packages (from pytest->ray==0.8.0.dev2) (0.18)
Requirement already satisfied: wcwidth in /opt/conda/lib/python3.6/site-packages (from pytest->ray==0.8.0.dev2) (0.1.7)
Requirement already satisfied: attrs>=17.4.0 in /opt/conda/lib/python3.6/site-packages (from pytest->ray==0.8.0.dev2) (18.1.0)
Requirement already satisfied: py>=1.5.0 in /opt/conda/lib/python3.6/site-packages (from pytest->ray==0.8.0.dev2) (1.5.3)
Requirement already satisfied: packaging in /opt/conda/lib/python3.6/site-packages (from pytest->ray==0.8.0.dev2) (17.1)
Requirement already satisfied: more-itertools>=4.0.0 in /opt/conda/lib/python3.6/site-packages (from pytest->ray==0.8.0.dev2) (4.1.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in /opt/conda/lib/python3.6/site-packages (from pytest->ray==0.8.0.dev2) (0.12.0)
Requirement already satisfied: atomicwrites>=1.0 in /opt/conda/lib/python3.6/site-packages (from pytest->ray==0.8.0.dev2) (1.3.0)
Requirement already satisfied: setuptools in /opt/conda/lib/python3.6/site-packages (from protobuf>=3.8.0->ray==0.8.0.dev2) (41.0.1)
Requirement already satisfied: zipp>=0.5 in /opt/conda/lib/python3.6/site-packages (from importlib-metadata>=0.12->pytest->ray==0.8.0.dev2) (0.5.2)
Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/lib/python3.6/site-packages (from packaging->pytest->ray==0.8.0.dev2) (2.2.0)
distributed 1.21.8 requires dask>=0.17.0, which is not installed.
distributed 1.21.8 requires msgpack, which is not installed.
blaze 0.11.3 requires dask[dataframe], which is not installed.
jupyterlab-server 1.0.0 has requirement jsonschema>=3.0.1, but you'll have jsonschema 2.6.0 which is incompatible.
Installing collected packages: ray
  Found existing installation: ray 0.8.0.dev2
    Uninstalling ray-0.8.0.dev2:
      Successfully uninstalled ray-0.8.0.dev2
  Running setup.py develop for ray
Successfully installed ray-0.8.0.dev2
You are using pip version 10.0.1, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
/ray/ci/jenkins_tests/perf_integration_tests /ray
+ popd
+ python -m pytest --benchmark-compare --benchmark-min-rounds=10 --benchmark-compare-fail=min:5% '--benchmark-columns=min, max, mean' /ray/ci/jenkins_tests/perf_integration_tests/../../../python/ray/tests/perf_integration_tests/test_perf_integration.py
Comparing against benchmarks from: Linux-CPython-3.6-64bit/0001_unversioned_20190713_225612.json
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-5.0.1, py-1.5.3, pluggy-0.12.0
benchmark: 3.2.2 (defaults: timer=time.perf_counter disable_gc=False min_rounds=10 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /ray/python
plugins: arraydiff-0.2, openfiles-0.3.0, doctestplus-0.1.3, benchmark-3.2.2, remotedata-0.3.1, timeout-1.3.3
collected 12 items

../../../python/ray/tests/perf_integration_tests/test_perf_integration.py . [  8%]
...........                                                              [100%]



Computing stats ...
Computing stats ... group 1/1
Computing stats ... group 1/1: min
Computing stats ... group 1/1: min (1/24)
Computing stats ... group 1/1: min (2/24)
Computing stats ... group 1/1: min (3/24)
Computing stats ... group 1/1: min (4/24)
Computing stats ... group 1/1: min (5/24)
Computing stats ... group 1/1: min (6/24)
Computing stats ... group 1/1: min (7/24)
Computing stats ... group 1/1: min (8/24)
Computing stats ... group 1/1: min (9/24)
Computing stats ... group 1/1: min (10/24)
Computing stats ... group 1/1: min (11/24)
Computing stats ... group 1/1: min (12/24)
Computing stats ... group 1/1: min (13/24)
Computing stats ... group 1/1: min (14/24)
Computing stats ... group 1/1: min (15/24)
Computing stats ... group 1/1: min (16/24)
Computing stats ... group 1/1: min (17/24)
Computing stats ... group 1/1: min (18/24)
Computing stats ... group 1/1: min (19/24)
Computing stats ... group 1/1: min (20/24)
Computing stats ... group 1/1: min (21/24)
Computing stats ... group 1/1: min (22/24)
Computing stats ... group 1/1: min (23/24)
Computing stats ... group 1/1: min (24/24)
Computing stats ... group 1/1: min (1/24)
Computing stats ... group 1/1: min (2/24)
Computing stats ... group 1/1: min (3/24)
Computing stats ... group 1/1: min (4/24)
Computing stats ... group 1/1: min (5/24)
Computing stats ... group 1/1: min (6/24)
Computing stats ... group 1/1: min (7/24)
Computing stats ... group 1/1: min (8/24)
Computing stats ... group 1/1: min (9/24)
Computing stats ... group 1/1: min (10/24)
Computing stats ... group 1/1: min (11/24)
Computing stats ... group 1/1: min (12/24)
Computing stats ... group 1/1: min (13/24)
Computing stats ... group 1/1: min (14/24)
Computing stats ... group 1/1: min (15/24)
Computing stats ... group 1/1: min (16/24)
Computing stats ... group 1/1: min (17/24)
Computing stats ... group 1/1: min (18/24)
Computing stats ... group 1/1: min (19/24)
Computing stats ... group 1/1: min (20/24)
Computing stats ... group 1/1: min (21/24)
Computing stats ... group 1/1: min (22/24)
Computing stats ... group 1/1: min (23/24)
Computing stats ... group 1/1: min (24/24)
Computing stats ... group 1/1: max
Computing stats ... group 1/1: max (1/24)
Computing stats ... group 1/1: max (2/24)
Computing stats ... group 1/1: max (3/24)
Computing stats ... group 1/1: max (4/24)
Computing stats ... group 1/1: max (5/24)
Computing stats ... group 1/1: max (6/24)
Computing stats ... group 1/1: max (7/24)
Computing stats ... group 1/1: max (8/24)
Computing stats ... group 1/1: max (9/24)
Computing stats ... group 1/1: max (10/24)
Computing stats ... group 1/1: max (11/24)
Computing stats ... group 1/1: max (12/24)
Computing stats ... group 1/1: max (13/24)
Computing stats ... group 1/1: max (14/24)
Computing stats ... group 1/1: max (15/24)
Computing stats ... group 1/1: max (16/24)
Computing stats ... group 1/1: max (17/24)
Computing stats ... group 1/1: max (18/24)
Computing stats ... group 1/1: max (19/24)
Computing stats ... group 1/1: max (20/24)
Computing stats ... group 1/1: max (21/24)
Computing stats ... group 1/1: max (22/24)
Computing stats ... group 1/1: max (23/24)
Computing stats ... group 1/1: max (24/24)
Computing stats ... group 1/1: max (1/24)
Computing stats ... group 1/1: max (2/24)
Computing stats ... group 1/1: max (3/24)
Computing stats ... group 1/1: max (4/24)
Computing stats ... group 1/1: max (5/24)
Computing stats ... group 1/1: max (6/24)
Computing stats ... group 1/1: max (7/24)
Computing stats ... group 1/1: max (8/24)
Computing stats ... group 1/1: max (9/24)
Computing stats ... group 1/1: max (10/24)
Computing stats ... group 1/1: max (11/24)
Computing stats ... group 1/1: max (12/24)
Computing stats ... group 1/1: max (13/24)
Computing stats ... group 1/1: max (14/24)
Computing stats ... group 1/1: max (15/24)
Computing stats ... group 1/1: max (16/24)
Computing stats ... group 1/1: max (17/24)
Computing stats ... group 1/1: max (18/24)
Computing stats ... group 1/1: max (19/24)
Computing stats ... group 1/1: max (20/24)
Computing stats ... group 1/1: max (21/24)
Computing stats ... group 1/1: max (22/24)
Computing stats ... group 1/1: max (23/24)
Computing stats ... group 1/1: max (24/24)
Computing stats ... group 1/1: mean
Computing stats ... group 1/1: mean (1/24)
Computing stats ... group 1/1: mean (2/24)
Computing stats ... group 1/1: mean (3/24)
Computing stats ... group 1/1: mean (4/24)
Computing stats ... group 1/1: mean (5/24)
Computing stats ... group 1/1: mean (6/24)
Computing stats ... group 1/1: mean (7/24)
Computing stats ... group 1/1: mean (8/24)
Computing stats ... group 1/1: mean (9/24)
Computing stats ... group 1/1: mean (10/24)
Computing stats ... group 1/1: mean (11/24)
Computing stats ... group 1/1: mean (12/24)
Computing stats ... group 1/1: mean (13/24)
Computing stats ... group 1/1: mean (14/24)
Computing stats ... group 1/1: mean (15/24)
Computing stats ... group 1/1: mean (16/24)
Computing stats ... group 1/1: mean (17/24)
Computing stats ... group 1/1: mean (18/24)
Computing stats ... group 1/1: mean (19/24)
Computing stats ... group 1/1: mean (20/24)
Computing stats ... group 1/1: mean (21/24)
Computing stats ... group 1/1: mean (22/24)
Computing stats ... group 1/1: mean (23/24)
Computing stats ... group 1/1: mean (24/24)
Computing stats ... group 1/1: mean (1/24)
Computing stats ... group 1/1: mean (2/24)
Computing stats ... group 1/1: mean (3/24)
Computing stats ... group 1/1: mean (4/24)
Computing stats ... group 1/1: mean (5/24)
Computing stats ... group 1/1: mean (6/24)
Computing stats ... group 1/1: mean (7/24)
Computing stats ... group 1/1: mean (8/24)
Computing stats ... group 1/1: mean (9/24)
Computing stats ... group 1/1: mean (10/24)
Computing stats ... group 1/1: mean (11/24)
Computing stats ... group 1/1: mean (12/24)
Computing stats ... group 1/1: mean (13/24)
Computing stats ... group 1/1: mean (14/24)
Computing stats ... group 1/1: mean (15/24)
Computing stats ... group 1/1: mean (16/24)
Computing stats ... group 1/1: mean (17/24)
Computing stats ... group 1/1: mean (18/24)
Computing stats ... group 1/1: mean (19/24)
Computing stats ... group 1/1: mean (20/24)
Computing stats ... group 1/1: mean (21/24)
Computing stats ... group 1/1: mean (22/24)
Computing stats ... group 1/1: mean (23/24)
Computing stats ... group 1/1: mean (24/24)
Computing stats ... group 1/1: median
Computing stats ... group 1/1: median (1/24)
Computing stats ... group 1/1: median (2/24)
Computing stats ... group 1/1: median (3/24)
Computing stats ... group 1/1: median (4/24)
Computing stats ... group 1/1: median (5/24)
Computing stats ... group 1/1: median (6/24)
Computing stats ... group 1/1: median (7/24)
Computing stats ... group 1/1: median (8/24)
Computing stats ... group 1/1: median (9/24)
Computing stats ... group 1/1: median (10/24)
Computing stats ... group 1/1: median (11/24)
Computing stats ... group 1/1: median (12/24)
Computing stats ... group 1/1: median (13/24)
Computing stats ... group 1/1: median (14/24)
Computing stats ... group 1/1: median (15/24)
Computing stats ... group 1/1: median (16/24)
Computing stats ... group 1/1: median (17/24)
Computing stats ... group 1/1: median (18/24)
Computing stats ... group 1/1: median (19/24)
Computing stats ... group 1/1: median (20/24)
Computing stats ... group 1/1: median (21/24)
Computing stats ... group 1/1: median (22/24)
Computing stats ... group 1/1: median (23/24)
Computing stats ... group 1/1: median (24/24)
Computing stats ... group 1/1: median (1/24)
Computing stats ... group 1/1: median (2/24)
Computing stats ... group 1/1: median (3/24)
Computing stats ... group 1/1: median (4/24)
Computing stats ... group 1/1: median (5/24)
Computing stats ... group 1/1: median (6/24)
Computing stats ... group 1/1: median (7/24)
Computing stats ... group 1/1: median (8/24)
Computing stats ... group 1/1: median (9/24)
Computing stats ... group 1/1: median (10/24)
Computing stats ... group 1/1: median (11/24)
Computing stats ... group 1/1: median (12/24)
Computing stats ... group 1/1: median (13/24)
Computing stats ... group 1/1: median (14/24)
Computing stats ... group 1/1: median (15/24)
Computing stats ... group 1/1: median (16/24)
Computing stats ... group 1/1: median (17/24)
Computing stats ... group 1/1: median (18/24)
Computing stats ... group 1/1: median (19/24)
Computing stats ... group 1/1: median (20/24)
Computing stats ... group 1/1: median (21/24)
Computing stats ... group 1/1: median (22/24)
Computing stats ... group 1/1: median (23/24)
Computing stats ... group 1/1: median (24/24)
Computing stats ... group 1/1: iqr
Computing stats ... group 1/1: iqr (1/24)
Computing stats ... group 1/1: iqr (2/24)
Computing stats ... group 1/1: iqr (3/24)
Computing stats ... group 1/1: iqr (4/24)
Computing stats ... group 1/1: iqr (5/24)
Computing stats ... group 1/1: iqr (6/24)
Computing stats ... group 1/1: iqr (7/24)
Computing stats ... group 1/1: iqr (8/24)
Computing stats ... group 1/1: iqr (9/24)
Computing stats ... group 1/1: iqr (10/24)
Computing stats ... group 1/1: iqr (11/24)
Computing stats ... group 1/1: iqr (12/24)
Computing stats ... group 1/1: iqr (13/24)
Computing stats ... group 1/1: iqr (14/24)
Computing stats ... group 1/1: iqr (15/24)
Computing stats ... group 1/1: iqr (16/24)
Computing stats ... group 1/1: iqr (17/24)
Computing stats ... group 1/1: iqr (18/24)
Computing stats ... group 1/1: iqr (19/24)
Computing stats ... group 1/1: iqr (20/24)
Computing stats ... group 1/1: iqr (21/24)
Computing stats ... group 1/1: iqr (22/24)
Computing stats ... group 1/1: iqr (23/24)
Computing stats ... group 1/1: iqr (24/24)
Computing stats ... group 1/1: iqr (1/24)
Computing stats ... group 1/1: iqr (2/24)
Computing stats ... group 1/1: iqr (3/24)
Computing stats ... group 1/1: iqr (4/24)
Computing stats ... group 1/1: iqr (5/24)
Computing stats ... group 1/1: iqr (6/24)
Computing stats ... group 1/1: iqr (7/24)
Computing stats ... group 1/1: iqr (8/24)
Computing stats ... group 1/1: iqr (9/24)
Computing stats ... group 1/1: iqr (10/24)
Computing stats ... group 1/1: iqr (11/24)
Computing stats ... group 1/1: iqr (12/24)
Computing stats ... group 1/1: iqr (13/24)
Computing stats ... group 1/1: iqr (14/24)
Computing stats ... group 1/1: iqr (15/24)
Computing stats ... group 1/1: iqr (16/24)
Computing stats ... group 1/1: iqr (17/24)
Computing stats ... group 1/1: iqr (18/24)
Computing stats ... group 1/1: iqr (19/24)
Computing stats ... group 1/1: iqr (20/24)
Computing stats ... group 1/1: iqr (21/24)
Computing stats ... group 1/1: iqr (22/24)
Computing stats ... group 1/1: iqr (23/24)
Computing stats ... group 1/1: iqr (24/24)
Computing stats ... group 1/1: stddev
Computing stats ... group 1/1: stddev (1/24)
Computing stats ... group 1/1: stddev (2/24)
Computing stats ... group 1/1: stddev (3/24)
Computing stats ... group 1/1: stddev (4/24)
Computing stats ... group 1/1: stddev (5/24)
Computing stats ... group 1/1: stddev (6/24)
Computing stats ... group 1/1: stddev (7/24)
Computing stats ... group 1/1: stddev (8/24)
Computing stats ... group 1/1: stddev (9/24)
Computing stats ... group 1/1: stddev (10/24)
Computing stats ... group 1/1: stddev (11/24)
Computing stats ... group 1/1: stddev (12/24)
Computing stats ... group 1/1: stddev (13/24)
Computing stats ... group 1/1: stddev (14/24)
Computing stats ... group 1/1: stddev (15/24)
Computing stats ... group 1/1: stddev (16/24)
Computing stats ... group 1/1: stddev (17/24)
Computing stats ... group 1/1: stddev (18/24)
Computing stats ... group 1/1: stddev (19/24)
Computing stats ... group 1/1: stddev (20/24)
Computing stats ... group 1/1: stddev (21/24)
Computing stats ... group 1/1: stddev (22/24)
Computing stats ... group 1/1: stddev (23/24)
Computing stats ... group 1/1: stddev (24/24)
Computing stats ... group 1/1: stddev (1/24)
Computing stats ... group 1/1: stddev (2/24)
Computing stats ... group 1/1: stddev (3/24)
Computing stats ... group 1/1: stddev (4/24)
Computing stats ... group 1/1: stddev (5/24)
Computing stats ... group 1/1: stddev (6/24)
Computing stats ... group 1/1: stddev (7/24)
Computing stats ... group 1/1: stddev (8/24)
Computing stats ... group 1/1: stddev (9/24)
Computing stats ... group 1/1: stddev (10/24)
Computing stats ... group 1/1: stddev (11/24)
Computing stats ... group 1/1: stddev (12/24)
Computing stats ... group 1/1: stddev (13/24)
Computing stats ... group 1/1: stddev (14/24)
Computing stats ... group 1/1: stddev (15/24)
Computing stats ... group 1/1: stddev (16/24)
Computing stats ... group 1/1: stddev (17/24)
Computing stats ... group 1/1: stddev (18/24)
Computing stats ... group 1/1: stddev (19/24)
Computing stats ... group 1/1: stddev (20/24)
Computing stats ... group 1/1: stddev (21/24)
Computing stats ... group 1/1: stddev (22/24)
Computing stats ... group 1/1: stddev (23/24)
Computing stats ... group 1/1: stddev (24/24)
Computing stats ... group 1/1: ops
Computing stats ... group 1/1: ops (1/24)
Computing stats ... group 1/1: ops (2/24)
Computing stats ... group 1/1: ops (3/24)
Computing stats ... group 1/1: ops (4/24)
Computing stats ... group 1/1: ops (5/24)
Computing stats ... group 1/1: ops (6/24)
Computing stats ... group 1/1: ops (7/24)
Computing stats ... group 1/1: ops (8/24)
Computing stats ... group 1/1: ops (9/24)
Computing stats ... group 1/1: ops (10/24)
Computing stats ... group 1/1: ops (11/24)
Computing stats ... group 1/1: ops (12/24)
Computing stats ... group 1/1: ops (13/24)
Computing stats ... group 1/1: ops (14/24)
Computing stats ... group 1/1: ops (15/24)
Computing stats ... group 1/1: ops (16/24)
Computing stats ... group 1/1: ops (17/24)
Computing stats ... group 1/1: ops (18/24)
Computing stats ... group 1/1: ops (19/24)
Computing stats ... group 1/1: ops (20/24)
Computing stats ... group 1/1: ops (21/24)
Computing stats ... group 1/1: ops (22/24)
Computing stats ... group 1/1: ops (23/24)
Computing stats ... group 1/1: ops (24/24)
Computing stats ... group 1/1: ops (1/24)
Computing stats ... group 1/1: ops (2/24)
Computing stats ... group 1/1: ops (3/24)
Computing stats ... group 1/1: ops (4/24)
Computing stats ... group 1/1: ops (5/24)
Computing stats ... group 1/1: ops (6/24)
Computing stats ... group 1/1: ops (7/24)
Computing stats ... group 1/1: ops (8/24)
Computing stats ... group 1/1: ops (9/24)
Computing stats ... group 1/1: ops (10/24)
Computing stats ... group 1/1: ops (11/24)
Computing stats ... group 1/1: ops (12/24)
Computing stats ... group 1/1: ops (13/24)
Computing stats ... group 1/1: ops (14/24)
Computing stats ... group 1/1: ops (15/24)
Computing stats ... group 1/1: ops (16/24)
Computing stats ... group 1/1: ops (17/24)
Computing stats ... group 1/1: ops (18/24)
Computing stats ... group 1/1: ops (19/24)
Computing stats ... group 1/1: ops (20/24)
Computing stats ... group 1/1: ops (21/24)
Computing stats ... group 1/1: ops (22/24)
Computing stats ... group 1/1: ops (23/24)
Computing stats ... group 1/1: ops (24/24)
Computing stats ... group 1/1: ops: outliers
Computing stats ... group 1/1: ops: outliers (1/24)
Computing stats ... group 1/1: ops: outliers (2/24)
Computing stats ... group 1/1: ops: outliers (3/24)
Computing stats ... group 1/1: ops: outliers (4/24)
Computing stats ... group 1/1: ops: outliers (5/24)
Computing stats ... group 1/1: ops: outliers (6/24)
Computing stats ... group 1/1: ops: outliers (7/24)
Computing stats ... group 1/1: ops: outliers (8/24)
Computing stats ... group 1/1: ops: outliers (9/24)
Computing stats ... group 1/1: ops: outliers (10/24)
Computing stats ... group 1/1: ops: outliers (11/24)
Computing stats ... group 1/1: ops: outliers (12/24)
Computing stats ... group 1/1: ops: outliers (13/24)
Computing stats ... group 1/1: ops: outliers (14/24)
Computing stats ... group 1/1: ops: outliers (15/24)
Computing stats ... group 1/1: ops: outliers (16/24)
Computing stats ... group 1/1: ops: outliers (17/24)
Computing stats ... group 1/1: ops: outliers (18/24)
Computing stats ... group 1/1: ops: outliers (19/24)
Computing stats ... group 1/1: ops: outliers (20/24)
Computing stats ... group 1/1: ops: outliers (21/24)
Computing stats ... group 1/1: ops: outliers (22/24)
Computing stats ... group 1/1: ops: outliers (23/24)
Computing stats ... group 1/1: ops: outliers (24/24)
Computing stats ... group 1/1: ops: rounds
Computing stats ... group 1/1: ops: rounds (1/24)
Computing stats ... group 1/1: ops: rounds (2/24)
Computing stats ... group 1/1: ops: rounds (3/24)
Computing stats ... group 1/1: ops: rounds (4/24)
Computing stats ... group 1/1: ops: rounds (5/24)
Computing stats ... group 1/1: ops: rounds (6/24)
Computing stats ... group 1/1: ops: rounds (7/24)
Computing stats ... group 1/1: ops: rounds (8/24)
Computing stats ... group 1/1: ops: rounds (9/24)
Computing stats ... group 1/1: ops: rounds (10/24)
Computing stats ... group 1/1: ops: rounds (11/24)
Computing stats ... group 1/1: ops: rounds (12/24)
Computing stats ... group 1/1: ops: rounds (13/24)
Computing stats ... group 1/1: ops: rounds (14/24)
Computing stats ... group 1/1: ops: rounds (15/24)
Computing stats ... group 1/1: ops: rounds (16/24)
Computing stats ... group 1/1: ops: rounds (17/24)
Computing stats ... group 1/1: ops: rounds (18/24)
Computing stats ... group 1/1: ops: rounds (19/24)
Computing stats ... group 1/1: ops: rounds (20/24)
Computing stats ... group 1/1: ops: rounds (21/24)
Computing stats ... group 1/1: ops: rounds (22/24)
Computing stats ... group 1/1: ops: rounds (23/24)
Computing stats ... group 1/1: ops: rounds (24/24)
Computing stats ... group 1/1: ops: iterations
Computing stats ... group 1/1: ops: iterations (1/24)
Computing stats ... group 1/1: ops: iterations (2/24)
Computing stats ... group 1/1: ops: iterations (3/24)
Computing stats ... group 1/1: ops: iterations (4/24)
Computing stats ... group 1/1: ops: iterations (5/24)
Computing stats ... group 1/1: ops: iterations (6/24)
Computing stats ... group 1/1: ops: iterations (7/24)
Computing stats ... group 1/1: ops: iterations (8/24)
Computing stats ... group 1/1: ops: iterations (9/24)
Computing stats ... group 1/1: ops: iterations (10/24)
Computing stats ... group 1/1: ops: iterations (11/24)
Computing stats ... group 1/1: ops: iterations (12/24)
Computing stats ... group 1/1: ops: iterations (13/24)
Computing stats ... group 1/1: ops: iterations (14/24)
Computing stats ... group 1/1: ops: iterations (15/24)
Computing stats ... group 1/1: ops: iterations (16/24)
Computing stats ... group 1/1: ops: iterations (17/24)
Computing stats ... group 1/1: ops: iterations (18/24)
Computing stats ... group 1/1: ops: iterations (19/24)
Computing stats ... group 1/1: ops: iterations (20/24)
Computing stats ... group 1/1: ops: iterations (21/24)
Computing stats ... group 1/1: ops: iterations (22/24)
Computing stats ... group 1/1: ops: iterations (23/24)
Computing stats ... group 1/1: ops: iterations (24/24)
----------------------------------------------------- benchmark: 24 tests -----------------------------------------------------
Name (time in ms)                                                 Min                     Max                    Mean          
-------------------------------------------------------------------------------------------------------------------------------
test_transfer_performance[1000-500] (0001_unversi)            78.0780 (1.0)           98.9794 (1.0)           88.3041 (1.0)    
test_transfer_performance[1000-5000] (NOW)                    85.9641 (1.10)         104.3217 (1.05)          90.1999 (1.02)   
test_transfer_performance[1000-500] (NOW)                     92.5152 (1.18)         111.6823 (1.13)         102.0793 (1.16)   
test_transfer_performance[1000-5000] (0001_unversi)          103.9944 (1.33)         113.6821 (1.15)         108.2724 (1.23)   
test_task_forward[1000_tasks] (NOW)                          317.3376 (4.06)         373.0135 (3.77)         346.7846 (3.93)   
test_task_forward[1000_tasks] (0001_unversi)                 320.1597 (4.10)         377.1385 (3.81)         351.0253 (3.98)   
test_transfer_performance[10000-500] (NOW)                   694.1766 (8.89)         841.4052 (8.50)         727.0985 (8.23)   
test_transfer_performance[10000-500] (0001_unversi)          705.7309 (9.04)         803.4605 (8.12)         760.8389 (8.62)   
test_transfer_performance[10000-5000] (0001_unversi)         722.6228 (9.26)         795.0024 (8.03)         765.3734 (8.67)   
test_transfer_performance[10000-5000] (NOW)                  752.1134 (9.63)         881.5791 (8.91)         803.0683 (9.09)   
test_task_forward[10000_tasks] (0001_unversi)              2,942.9156 (37.69)      3,326.9708 (33.61)      3,073.8600 (34.81)  
test_task_forward[10000_tasks] (NOW)                       3,011.2267 (38.57)      3,236.2038 (32.70)      3,121.9882 (35.35)  
test_task_submission[100_tasks] (0001_unversi)            22,535.5087 (288.63)    22,826.6716 (230.62)    22,624.1203 (256.21) 
test_task_submission[1000_tasks] (NOW)                    22,850.2988 (292.66)    23,291.3857 (235.32)    23,062.9762 (261.18) 
test_task_submission[100_tasks] (NOW)                     22,871.3187 (292.93)    23,836.5403 (240.82)    23,100.9997 (261.61) 
test_task_submission[10000_tasks] (NOW)                   22,893.0700 (293.21)    24,265.0747 (245.15)    23,770.1878 (269.19) 
test_task_submission[1000_tasks] (0001_unversi)           23,234.5893 (297.58)    24,211.8219 (244.61)    23,567.0319 (266.88) 
test_task_submission[10000_tasks] (0001_unversi)          23,285.6245 (298.24)    23,874.6958 (241.21)    23,551.8852 (266.71) 
test_task_submission[10_tasks] (0001_unversi)             29,259.1614 (374.74)    30,245.1633 (305.57)    29,756.6330 (336.98) 
test_task_submission[10_tasks] (NOW)                      29,619.5004 (379.36)    30,295.1524 (306.08)    29,917.7489 (338.80) 
test_task_submission[1_tasks] (NOW)                      106,572.6352 (>1000.0)  132,014.3307 (>1000.0)  119,434.8921 (>1000.0)
test_task_submission[1_tasks] (0001_unversi)             112,663.9791 (>1000.0)  135,063.5094 (>1000.0)  129,188.0302 (>1000.0)
test_task_submission[100000_tasks] (NOW)                 148,312.0674 (>1000.0)  222,341.2600 (>1000.0)  158,140.3462 (>1000.0)
test_task_submission[100000_tasks] (0001_unversi)        149,491.9673 (>1000.0)  201,440.6912 (>1000.0)  161,452.0020 (>1000.0)
-------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

--------------------------------------------------------------------------------
Performance has regressed:
	test_transfer_performance[1000-500] (0001_unversi) - Field 'min' has failed PercentageRegressionCheck: 18.490819441 > 5.000000000
--------------------------------------------------------------------------------
ERROR: Performance has regressed.

Build step 'Execute shell' marked build as failure
An attempt to send an e-mail to empty list of recipients, ignored.
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/Ray-Perf-Integration-PRB/1662/
Test FAILed.
Finished: FAILURE