SuccessConsole Output

Skipping 9,843 KB.. Full Log
eating DockerContainerManager 
 [integration_py36_many_apps_many_models] 19-07-09:00:26:22 INFO     [test_utils.py:112] Cleaning up Docker cluster many-app-2339 
 [integration_py36_mxnet] 19-07-09:00:26:24 INFO     [test_utils.py:107] Creating DockerContainerManager 
 [integration_py36_mxnet] 19-07-09:00:26:24 INFO     [test_utils.py:112] Cleaning up Docker cluster mxnet-4142 
 [integration_py36_admin_unit_test] 19-07-09:00:26:25 INFO     [test_utils.py:107] Creating DockerContainerManager 
 [integration_py36_admin_unit_test] 19-07-09:00:26:25 INFO     [test_utils.py:112] Cleaning up Docker cluster admin-test-cluster-239 
 [integration_py36_docker_metric] 19-07-09:00:26:26 INFO     [docker_container_manager.py:433] [metric-d-36641] Found 0 replicas for simple-example:1. Adding 2 
 [integration_py36_multi_tenancy] 19-07-09:00:26:28 INFO     [clipper_admin.py:1424] [multi-tenancy-1-2623] Stopped all Clipper cluster and all model containers 
 [integration_py36_admin_unit_test] 19-07-09:00:26:29 INFO     [clipper_admin.py:1424] [admin-test-cluster-239] Stopped all Clipper cluster and all model containers 
 [integration_py36_admin_unit_test] ok 
 [integration_py36_admin_unit_test] test_unregister_application_correct (__main__.ClipperManagerTestCaseShort) ... 19-07-09:00:26:29 INFO     [test_utils.py:107] Creating DockerContainerManager 
 [integration_py36_admin_unit_test] 19-07-09:00:26:29 INFO     [test_utils.py:128] Starting up Docker cluster admin-test-cluster-3579 
 [integration_py36_admin_unit_test] 19-07-09:00:26:29 INFO     [test_utils.py:143] Starting Clipper 
 [integration_py36_admin_unit_test] 19-07-09:00:26:29 INFO     [docker_container_manager.py:192] [admin-test-cluster-3579] Starting managed Redis instance in Docker 
 [integration_py36_mxnet] 19-07-09:00:26:31 INFO     [clipper_admin.py:1424] [mxnet-4142] Stopped all Clipper cluster and all model containers 
 [integration_py36_mxnet] 19-07-08:17:26:31 INFO     [retry_with_timeout.py:90] Success! 
===== finished: integration_py36_mxnet =====
 [integration_py36_many_apps_many_models] 19-07-09:00:26:31 INFO     [clipper_admin.py:1424] [many-app-2339] Stopped all Clipper cluster and all model containers 
 [integration_py36_many_apps_many_models] 19-07-08:17:26:32 INFO     [retry_with_timeout.py:90] Success! 
===== finished: integration_py36_many_apps_many_models =====
 [integration_py36_docker_metric] 19-07-09:00:26:32 INFO     [clipper_admin.py:724] [metric-d-36641] Successfully registered model simple-example:1 
 [integration_py36_docker_metric] 19-07-09:00:26:32 INFO     [clipper_admin.py:642] [metric-d-36641] Done deploying model simple-example:1. 
 [integration_py36_docker_metric] 19-07-09:00:26:32 INFO     [clipper_admin.py:303] [metric-d-36641] Model simple-example is now linked to application simple-example 
 [integration_py36_admin_unit_test] 19-07-09:00:26:35 INFO     [docker_container_manager.py:294] [admin-test-cluster-3579] Metric Configuration Saved at /tmp/tmpk258504z.yml 
 [integration_py36_admin_unit_test] 19-07-09:00:26:36 INFO     [clipper_admin.py:162] [admin-test-cluster-3579] Clipper is running 
 [integration_py36_admin_unit_test] 19-07-09:00:26:37 INFO     [clipper_admin.py:236] [admin-test-cluster-3579] Application testapp was successfully registered 
 [integration_py36_admin_unit_test] 19-07-09:00:26:37 INFO     [clipper_admin.py:260] [admin-test-cluster-3579] Application testapp was successfully unregistered 
 [integration_py36_admin_unit_test] 19-07-09:00:26:37 INFO     [test_utils.py:107] Creating DockerContainerManager 
 [integration_py36_admin_unit_test] 19-07-09:00:26:37 INFO     [test_utils.py:112] Cleaning up Docker cluster admin-test-cluster-3579 
 [integration_py36_pyspark] 19-07-09:00:26:39 INFO     [deployer_utils.py:41] Saving function to /tmp/tmpfh_u7a3oclipper 
 [integration_py36_pyspark] 19-07-09:00:26:39 INFO     [deployer_utils.py:51] Serialized and supplied predict function 
 [integration_py36_pyspark] 19-07-09:00:26:39 INFO     [pyspark.py:234] Spark model saved 
 [integration_py36_pyspark] 19-07-09:00:26:39 INFO     [pyspark.py:248] Using Python 3.6 base image 
 [integration_py36_pyspark] 19-07-09:00:26:39 INFO     [clipper_admin.py:534] [spark-3398] Building model Docker image with model data from /tmp/tmpfh_u7a3oclipper 
 [integration_py36_pyspark] 19-07-09:00:26:39 INFO     [clipper_admin.py:539] [spark-3398] Step 1/2 : FROM clippertesting/pyspark36-container:c1b57bb7c9 
 [integration_py36_pyspark] 19-07-09:00:26:39 INFO     [clipper_admin.py:539] [spark-3398]  ---> 71634257229e 
 [integration_py36_pyspark] 19-07-09:00:26:39 INFO     [clipper_admin.py:539] [spark-3398] Step 2/2 : COPY /tmp/tmpfh_u7a3oclipper /model/ 
 [integration_py36_pyspark] 19-07-09:00:26:39 INFO     [clipper_admin.py:539] [spark-3398]  ---> 9e91edffbb56 
 [integration_py36_pyspark] 19-07-09:00:26:39 INFO     [clipper_admin.py:539] [spark-3398] Successfully built 9e91edffbb56 
 [integration_py36_pyspark] 19-07-09:00:26:39 INFO     [clipper_admin.py:539] [spark-3398] Successfully tagged spark-3398-pyspark-model:3 
 [integration_py36_pyspark] 19-07-09:00:26:39 INFO     [clipper_admin.py:541] [spark-3398] Pushing model Docker image to spark-3398-pyspark-model:3 
 [integration_py36_admin_unit_test] 19-07-09:00:26:40 INFO     [clipper_admin.py:1424] [admin-test-cluster-3579] Stopped all Clipper cluster and all model containers 
 [integration_py36_admin_unit_test] ok 
 [integration_py36_admin_unit_test] test_query_specific_model_version (__main__.ClipperManagerTestCaseShort) ... 19-07-09:00:26:40 INFO     [test_utils.py:107] Creating DockerContainerManager 
 [integration_py36_admin_unit_test] 19-07-09:00:26:40 INFO     [test_utils.py:128] Starting up Docker cluster admin-test-cluster-4664 
 [integration_py36_admin_unit_test] 19-07-09:00:26:40 INFO     [test_utils.py:143] Starting Clipper 
 [integration_py36_admin_unit_test] 19-07-09:00:26:40 INFO     [docker_container_manager.py:192] [admin-test-cluster-4664] Starting managed Redis instance in Docker 
 [integration_py36_pyspark] 19-07-09:00:26:40 INFO     [docker_container_manager.py:433] [spark-3398] Found 0 replicas for pyspark-model:3. Adding 1 
 [integration_py36_pyspark] 19-07-09:00:26:41 INFO     [clipper_admin.py:724] [spark-3398] Successfully registered model pyspark-model:3 
 [integration_py36_pyspark] 19-07-09:00:26:41 INFO     [clipper_admin.py:642] [spark-3398] Done deploying model pyspark-model:3. 
 [integration_py36_docker_metric] 19-07-09:00:26:42 INFO     [clipper_metric_docker.py:99] Making 100 predictions using two model container; Should takes 50 seconds. 
 [integration_py36_admin_unit_test] 19-07-09:00:26:44 INFO     [docker_container_manager.py:294] [admin-test-cluster-4664] Metric Configuration Saved at /tmp/tmphqp26goq.yml 
 [integration_py36_admin_unit_test] 19-07-09:00:26:45 INFO     [clipper_admin.py:162] [admin-test-cluster-4664] Clipper is running 
 [integration_py36_admin_unit_test] 19-07-09:00:26:47 INFO     [clipper_admin.py:236] [admin-test-cluster-4664] Application testapp was successfully registered 
 [integration_py36_admin_unit_test] 19-07-09:00:26:47 INFO     [deployer_utils.py:41] Saving function to /tmp/tmp4zjdsotvclipper 
 [integration_py36_admin_unit_test] 19-07-09:00:26:47 INFO     [deployer_utils.py:51] Serialized and supplied predict function 
 [integration_py36_admin_unit_test] 19-07-09:00:26:47 INFO     [python.py:192] Python closure saved 
 [integration_py36_admin_unit_test] 19-07-09:00:26:47 INFO     [python.py:206] Using Python 3.6 base image 
 [integration_py36_admin_unit_test] 19-07-09:00:26:47 INFO     [clipper_admin.py:534] [admin-test-cluster-4664] Building model Docker image with model data from /tmp/tmp4zjdsotvclipper 
 [integration_py36_admin_unit_test] 19-07-09:00:26:47 INFO     [clipper_admin.py:539] [admin-test-cluster-4664] Step 1/2 : FROM clippertesting/python36-closure-container:c1b57bb7c9 
 [integration_py36_admin_unit_test] 19-07-09:00:26:47 INFO     [clipper_admin.py:539] [admin-test-cluster-4664]  ---> a1b91d6ce362 
 [integration_py36_admin_unit_test] 19-07-09:00:26:47 INFO     [clipper_admin.py:539] [admin-test-cluster-4664] Step 2/2 : COPY /tmp/tmp4zjdsotvclipper /model/ 
 [integration_py36_admin_unit_test] 19-07-09:00:26:47 INFO     [clipper_admin.py:539] [admin-test-cluster-4664]  ---> 356e323875b6 
 [integration_py36_admin_unit_test] 19-07-09:00:26:47 INFO     [clipper_admin.py:539] [admin-test-cluster-4664] Successfully built 356e323875b6 
 [integration_py36_admin_unit_test] 19-07-09:00:26:47 INFO     [clipper_admin.py:539] [admin-test-cluster-4664] Successfully tagged admin-test-cluster-4664-testmodel:v1 
 [integration_py36_admin_unit_test] 19-07-09:00:26:47 INFO     [clipper_admin.py:541] [admin-test-cluster-4664] Pushing model Docker image to admin-test-cluster-4664-testmodel:v1 
 [integration_py36_admin_unit_test] 19-07-09:00:26:51 INFO     [docker_container_manager.py:433] [admin-test-cluster-4664] Found 0 replicas for testmodel:v1. Adding 1 
 [integration_py36_admin_unit_test] 19-07-09:00:26:52 INFO     [clipper_admin.py:724] [admin-test-cluster-4664] Successfully registered model testmodel:v1 
 [integration_py36_admin_unit_test] 19-07-09:00:26:52 INFO     [clipper_admin.py:642] [admin-test-cluster-4664] Done deploying model testmodel:v1. 
 [integration_py36_admin_unit_test] 19-07-09:00:26:52 INFO     [clipper_admin.py:303] [admin-test-cluster-4664] Model testmodel is now linked to application testapp 
 [integration_py36_pyspark] 19-07-09:00:27:11 INFO     [clipper_admin.py:236] [spark-3398] Application easy-register-app-model was successfully registered 
 [integration_py36_pyspark] 19-07-09:00:27:11 INFO     [deployer_utils.py:41] Saving function to /tmp/tmp190ehes1clipper 
 [integration_py36_pyspark] 19-07-09:00:27:11 INFO     [deployer_utils.py:51] Serialized and supplied predict function 
 [integration_py36_pyspark] 19-07-09:00:27:12 INFO     [pyspark.py:234] Spark model saved 
 [integration_py36_pyspark] 19-07-09:00:27:12 INFO     [pyspark.py:248] Using Python 3.6 base image 
 [integration_py36_pyspark] 19-07-09:00:27:12 INFO     [clipper_admin.py:534] [spark-3398] Building model Docker image with model data from /tmp/tmp190ehes1clipper 
 [integration_py36_multi_tenancy] 19-07-09:00:27:12 INFO     [clipper_admin.py:1424] [multi-tenancy-2-7379] Stopped all Clipper cluster and all model containers 
 [integration_py36_multi_tenancy] '{"query_id":0,"output":0.6000000000000001,"default":false}', 4.951000 ms 
 [integration_py36_multi_tenancy] '{"query_id":0,"output":0.6000000000000001,"default":false}', 4.861000 ms 
 [integration_py36_pyspark] 19-07-09:00:27:12 INFO     [clipper_admin.py:539] [spark-3398] Step 1/2 : FROM clippertesting/pyspark36-container:c1b57bb7c9 
 [integration_py36_pyspark] 19-07-09:00:27:12 INFO     [clipper_admin.py:539] [spark-3398]  ---> 71634257229e 
 [integration_py36_pyspark] 19-07-09:00:27:12 INFO     [clipper_admin.py:539] [spark-3398] Step 2/2 : COPY /tmp/tmp190ehes1clipper /model/ 
 [integration_py36_pyspark] 19-07-09:00:27:12 INFO     [clipper_admin.py:539] [spark-3398]  ---> 15a269f1b5fc 
 [integration_py36_pyspark] 19-07-09:00:27:12 INFO     [clipper_admin.py:539] [spark-3398] Successfully built 15a269f1b5fc 
 [integration_py36_pyspark] 19-07-09:00:27:12 INFO     [clipper_admin.py:539] [spark-3398] Successfully tagged spark-3398-easy-register-app-model:1 
 [integration_py36_pyspark] 19-07-09:00:27:12 INFO     [clipper_admin.py:541] [spark-3398] Pushing model Docker image to spark-3398-easy-register-app-model:1 
 [integration_py36_multi_tenancy] 19-07-08:17:27:12 INFO     [retry_with_timeout.py:90] Success! 
===== finished: integration_py36_multi_tenancy =====
 [integration_py36_pyspark] 19-07-09:00:27:13 INFO     [docker_container_manager.py:433] [spark-3398] Found 0 replicas for easy-register-app-model:1. Adding 1 
 [integration_py36_pyspark] 19-07-09:00:27:14 INFO     [clipper_admin.py:724] [spark-3398] Successfully registered model easy-register-app-model:1 
 [integration_py36_pyspark] 19-07-09:00:27:14 INFO     [clipper_admin.py:642] [spark-3398] Done deploying model easy-register-app-model:1. 
 [integration_py36_pyspark] 19-07-09:00:27:14 INFO     [clipper_admin.py:303] [spark-3398] Model easy-register-app-model is now linked to application easy-register-app-model 
 [integration_py36_admin_unit_test] 19-07-09:00:27:22 INFO     [deployer_utils.py:41] Saving function to /tmp/tmp14__qel3clipper 
 [integration_py36_admin_unit_test] 19-07-09:00:27:22 INFO     [deployer_utils.py:51] Serialized and supplied predict function 
 [integration_py36_admin_unit_test] 19-07-09:00:27:22 INFO     [python.py:192] Python closure saved 
 [integration_py36_admin_unit_test] 19-07-09:00:27:22 INFO     [python.py:206] Using Python 3.6 base image 
 [integration_py36_admin_unit_test] 19-07-09:00:27:22 INFO     [clipper_admin.py:534] [admin-test-cluster-4664] Building model Docker image with model data from /tmp/tmp14__qel3clipper 
 [integration_py36_admin_unit_test] 19-07-09:00:27:22 INFO     [clipper_admin.py:539] [admin-test-cluster-4664] Step 1/2 : FROM clippertesting/python36-closure-container:c1b57bb7c9 
 [integration_py36_admin_unit_test] 19-07-09:00:27:22 INFO     [clipper_admin.py:539] [admin-test-cluster-4664]  ---> a1b91d6ce362 
 [integration_py36_admin_unit_test] 19-07-09:00:27:22 INFO     [clipper_admin.py:539] [admin-test-cluster-4664] Step 2/2 : COPY /tmp/tmp14__qel3clipper /model/ 
 [integration_py36_admin_unit_test] 19-07-09:00:27:22 INFO     [clipper_admin.py:539] [admin-test-cluster-4664]  ---> 08c1798cbb64 
 [integration_py36_admin_unit_test] 19-07-09:00:27:22 INFO     [clipper_admin.py:539] [admin-test-cluster-4664] Successfully built 08c1798cbb64 
 [integration_py36_admin_unit_test] 19-07-09:00:27:22 INFO     [clipper_admin.py:539] [admin-test-cluster-4664] Successfully tagged admin-test-cluster-4664-testmodel:v2 
 [integration_py36_admin_unit_test] 19-07-09:00:27:22 INFO     [clipper_admin.py:541] [admin-test-cluster-4664] Pushing model Docker image to admin-test-cluster-4664-testmodel:v2 
 [integration_py36_admin_unit_test] 19-07-09:00:27:23 INFO     [docker_container_manager.py:433] [admin-test-cluster-4664] Found 0 replicas for testmodel:v2. Adding 1 
 [integration_py36_admin_unit_test] 19-07-09:00:27:24 INFO     [clipper_admin.py:724] [admin-test-cluster-4664] Successfully registered model testmodel:v2 
 [integration_py36_admin_unit_test] 19-07-09:00:27:24 INFO     [clipper_admin.py:642] [admin-test-cluster-4664] Done deploying model testmodel:v2. 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:105] Test 1: Checking status of 3 node exporter 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:57] Querying: http://localhost:44062/api/v1/targets 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:59] {'status': 'success', 'data': {'activeTargets': [{'discoveredLabels': {'__address__': 'query_frontend_exporter-51534:1390', '__metrics_path__': '/metrics', '__scheme__': 'http', 'job': 'query'}, 'labels': {'instance': 'query_frontend_exporter-51534:1390', 'job': 'query'}, 'scrapeUrl': 'http://query_frontend_exporter-51534:1390/metrics', 'lastError': '', 'lastScrape': '2019-07-09T00:27:29.022295944Z', 'health': 'up'}, {'discoveredLabels': {'__address__': 'simple-example_1-52298:1390', '__metrics_path__': '/metrics', '__scheme__': 'http', 'job': 'simple-example_1-52298'}, 'labels': {'instance': 'simple-example_1-52298:1390', 'job': 'simple-example_1-52298'}, 'scrapeUrl': 'http://simple-example_1-52298:1390/metrics', 'lastError': '', 'lastScrape': '2019-07-09T00:27:29.880089052Z', 'health': 'up'}, {'discoveredLabels': {'__address__': 'simple-example_1-74942:1390', '__metrics_path__': '/metrics', '__scheme__': 'http', 'job': 'simple-example_1-74942'}, 'labels': {'instance': 'simple-example_1-74942:1390', 'job': 'simple-example_1-74942'}, 'scrapeUrl': 'http://simple-example_1-74942:1390/metrics', 'lastError': '', 'lastScrape': '2019-07-09T00:27:32.719342864Z', 'health': 'up'}], 'droppedTargets': []}} 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:107] Test 1 Passed 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:109] Test 2: Checking Model Container Metrics 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:43] Querying: http://localhost:44062/api/v1/series?match[]=clipper_mc_pred_total 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:44] test why docker query is broken: 'http://localhost:44062/api/v1/series?match[]=clipper_mc_pred_total' 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:46] {'status': 'success', 'data': [{'__name__': 'clipper_mc_pred_total', 'instance': 'simple-example_1-52298:1390', 'job': 'simple-example_1-52298'}, {'__name__': 'clipper_mc_pred_total', 'instance': 'simple-example_1-74942:1390', 'job': 'simple-example_1-74942'}]} 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:43] Querying: http://localhost:44062/api/v1/series?match[]=clipper_mc_end_to_end_latency_ms_sum 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:44] test why docker query is broken: 'http://localhost:44062/api/v1/series?match[]=clipper_mc_end_to_end_latency_ms_sum' 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:46] {'status': 'success', 'data': [{'__name__': 'clipper_mc_end_to_end_latency_ms_sum', 'instance': 'simple-example_1-52298:1390', 'job': 'simple-example_1-52298'}, {'__name__': 'clipper_mc_end_to_end_latency_ms_sum', 'instance': 'simple-example_1-74942:1390', 'job': 'simple-example_1-74942'}]} 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:43] Querying: http://localhost:44062/api/v1/series?match[]=clipper_mc_recv_time_ms_sum 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:44] test why docker query is broken: 'http://localhost:44062/api/v1/series?match[]=clipper_mc_recv_time_ms_sum' 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:46] {'status': 'success', 'data': [{'__name__': 'clipper_mc_recv_time_ms_sum', 'instance': 'simple-example_1-52298:1390', 'job': 'simple-example_1-52298'}, {'__name__': 'clipper_mc_recv_time_ms_sum', 'instance': 'simple-example_1-74942:1390', 'job': 'simple-example_1-74942'}]} 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:43] Querying: http://localhost:44062/api/v1/series?match[]=clipper_mc_parse_time_ms_sum 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:44] test why docker query is broken: 'http://localhost:44062/api/v1/series?match[]=clipper_mc_parse_time_ms_sum' 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:46] {'status': 'success', 'data': [{'__name__': 'clipper_mc_parse_time_ms_sum', 'instance': 'simple-example_1-52298:1390', 'job': 'simple-example_1-52298'}, {'__name__': 'clipper_mc_parse_time_ms_sum', 'instance': 'simple-example_1-74942:1390', 'job': 'simple-example_1-74942'}]} 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:43] Querying: http://localhost:44062/api/v1/series?match[]=clipper_mc_handle_time_ms_sum 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:44] test why docker query is broken: 'http://localhost:44062/api/v1/series?match[]=clipper_mc_handle_time_ms_sum' 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:46] {'status': 'success', 'data': [{'__name__': 'clipper_mc_handle_time_ms_sum', 'instance': 'simple-example_1-52298:1390', 'job': 'simple-example_1-52298'}, {'__name__': 'clipper_mc_handle_time_ms_sum', 'instance': 'simple-example_1-74942:1390', 'job': 'simple-example_1-74942'}]} 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:120] Test 2 Passed 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [clipper_metric_docker.py:122] Metric Test Done, Cleaning up... 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [test_utils.py:107] Creating DockerContainerManager 
 [integration_py36_docker_metric] 19-07-09:00:27:33 INFO     [test_utils.py:112] Cleaning up Docker cluster metric-d-36641 
 [integration_py36_docker_metric] 19-07-09:00:27:37 INFO     [clipper_admin.py:1424] [metric-d-36641] Stopped all Clipper cluster and all model containers 
 [integration_py36_docker_metric] 19-07-08:17:27:37 INFO     [retry_with_timeout.py:90] Success! 
===== finished: integration_py36_docker_metric =====
 [integration_py36_pyspark] 19-07-09:00:27:39 INFO     [deployer_utils.py:41] Saving function to /tmp/tmpovfe91n6clipper 
 [integration_py36_pyspark] 19-07-09:00:27:39 INFO     [deployer_utils.py:51] Serialized and supplied predict function 
 [integration_py36_pyspark] 19-07-09:00:27:39 INFO     [pyspark.py:234] Spark model saved 
 [integration_py36_pyspark] 19-07-09:00:27:39 INFO     [pyspark.py:248] Using Python 3.6 base image 
 [integration_py36_pyspark] 19-07-09:00:27:39 INFO     [clipper_admin.py:534] [spark-3398] Building model Docker image with model data from /tmp/tmpovfe91n6clipper 
 [integration_py36_pyspark] 19-07-09:00:27:40 INFO     [clipper_admin.py:539] [spark-3398] Step 1/2 : FROM clippertesting/pyspark36-container:c1b57bb7c9 
 [integration_py36_pyspark] 19-07-09:00:27:40 INFO     [clipper_admin.py:539] [spark-3398]  ---> 71634257229e 
 [integration_py36_pyspark] 19-07-09:00:27:40 INFO     [clipper_admin.py:539] [spark-3398] Step 2/2 : COPY /tmp/tmpovfe91n6clipper /model/ 
 [integration_py36_pyspark] 19-07-09:00:27:40 INFO     [clipper_admin.py:539] [spark-3398]  ---> b1290a0d854f 
 [integration_py36_pyspark] 19-07-09:00:27:40 INFO     [clipper_admin.py:539] [spark-3398] Successfully built b1290a0d854f 
 [integration_py36_pyspark] 19-07-09:00:27:40 INFO     [clipper_admin.py:539] [spark-3398] Successfully tagged spark-3398-pyspark-model:4 
 [integration_py36_pyspark] 19-07-09:00:27:40 INFO     [clipper_admin.py:541] [spark-3398] Pushing model Docker image to spark-3398-pyspark-model:4 
 [integration_py36_pyspark] 19-07-09:00:27:41 INFO     [docker_container_manager.py:433] [spark-3398] Found 0 replicas for pyspark-model:4. Adding 1 
 [integration_py36_pyspark] 19-07-09:00:27:42 INFO     [clipper_admin.py:724] [spark-3398] Successfully registered model pyspark-model:4 
 [integration_py36_pyspark] 19-07-09:00:27:42 INFO     [clipper_admin.py:642] [spark-3398] Done deploying model pyspark-model:4. 
 [integration_py36_pyspark] 19-07-09:00:28:13 INFO     [test_utils.py:107] Creating DockerContainerManager 
 [integration_py36_pyspark] 19-07-09:00:28:13 INFO     [test_utils.py:112] Cleaning up Docker cluster spark-3398 
 [integration_py36_pyspark] 19-07-09:00:28:18 INFO     [clipper_admin.py:1424] [spark-3398] Stopped all Clipper cluster and all model containers 
 [integration_py36_pyspark] 19-07-08:17:28:18 INFO     [retry_with_timeout.py:90] Success! 
===== finished: integration_py36_pyspark =====
 [integration_py36_admin_unit_test] 19-07-09:00:28:24 INFO     [clipper_admin_tests.py:564] {"error":"Query processing error","cause":"Requested version: skjfhkdjshfjksdhkjf does not exist for model: testmodel"} 
 [integration_py36_admin_unit_test] 19-07-09:00:28:24 INFO     [test_utils.py:107] Creating DockerContainerManager 
 [integration_py36_admin_unit_test] 19-07-09:00:28:24 INFO     [test_utils.py:112] Cleaning up Docker cluster admin-test-cluster-4664 
 [integration_py36_admin_unit_test] 19-07-09:00:28:27 INFO     [clipper_admin.py:1424] [admin-test-cluster-4664] Stopped all Clipper cluster and all model containers 
 [integration_py36_admin_unit_test] ok 
 [integration_py36_admin_unit_test] 19-07-09:00:28:27 INFO     [test_utils.py:107] Creating DockerContainerManager 
 [integration_py36_admin_unit_test] 19-07-09:00:28:27 INFO     [test_utils.py:128] Starting up Docker cluster admin-l-42930 
 [integration_py36_admin_unit_test] 19-07-09:00:28:27 INFO     [test_utils.py:143] Starting Clipper 
 [integration_py36_admin_unit_test] 19-07-09:00:28:28 INFO     [docker_container_manager.py:192] [admin-l-42930] Starting managed Redis instance in Docker 
 [integration_py36_admin_unit_test] 19-07-09:00:28:32 INFO     [docker_container_manager.py:294] [admin-l-42930] Metric Configuration Saved at /tmp/tmpq_1m2htj.yml 
 [integration_py36_admin_unit_test] 19-07-09:00:28:33 INFO     [clipper_admin.py:162] [admin-l-42930] Clipper is running 
 [integration_py36_admin_unit_test] 19-07-09:00:28:34 INFO     [clipper_admin.py:236] [admin-l-42930] Application app3 was successfully registered 
 [integration_py36_admin_unit_test] 19-07-09:00:28:34 INFO     [clipper_admin.py:236] [admin-l-42930] Application app4 was successfully registered 
 [integration_py36_admin_unit_test] 19-07-09:00:28:34 INFO     [clipper_admin.py:236] [admin-l-42930] Application app5 was successfully registered 
 [integration_py36_admin_unit_test] 19-07-09:00:28:34 INFO     [clipper_admin.py:236] [admin-l-42930] Application app6 was successfully registered 
 [integration_py36_admin_unit_test] 19-07-09:00:28:34 INFO     [clipper_admin.py:236] [admin-l-42930] Application app7 was successfully registered 
 [integration_py36_admin_unit_test] test_remove_inactive_container (__main__.ClipperManagerTestCaseLong) ... 19-07-09:00:28:34 INFO     [clipper_admin.py:534] [admin-l-42930] Building model Docker image with model data from /tmp/clipper/tmp320pf9dl 
 [integration_py36_admin_unit_test] 19-07-09:00:28:34 INFO     [clipper_admin.py:539] [admin-l-42930] Step 1/2 : FROM clippertesting/noop-container:c1b57bb7c9 
 [integration_py36_admin_unit_test] 19-07-09:00:28:34 INFO     [clipper_admin.py:539] [admin-l-42930]  ---> 8435f7c6b4ac 
 [integration_py36_admin_unit_test] 19-07-09:00:28:34 INFO     [clipper_admin.py:539] [admin-l-42930] Step 2/2 : COPY /tmp/clipper/tmp320pf9dl /model/ 
 [integration_py36_admin_unit_test] 19-07-09:00:28:34 INFO     [clipper_admin.py:539] [admin-l-42930]  ---> Using cache 
 [integration_py36_admin_unit_test] 19-07-09:00:28:34 INFO     [clipper_admin.py:539] [admin-l-42930]  ---> 88bf956e4b97 
 [integration_py36_admin_unit_test] 19-07-09:00:28:34 INFO     [clipper_admin.py:539] [admin-l-42930] Successfully built 88bf956e4b97 
 [integration_py36_admin_unit_test] 19-07-09:00:28:34 INFO     [clipper_admin.py:539] [admin-l-42930] Successfully tagged admin-l-42930-m8:1 
 [integration_py36_admin_unit_test] 19-07-09:00:28:34 INFO     [clipper_admin.py:541] [admin-l-42930] Pushing model Docker image to admin-l-42930-m8:1 
 [integration_py36_admin_unit_test] 19-07-09:00:28:35 INFO     [docker_container_manager.py:433] [admin-l-42930] Found 0 replicas for m8:1. Adding 2 
 [integration_py36_admin_unit_test] 19-07-09:00:28:37 INFO     [clipper_admin.py:724] [admin-l-42930] Successfully registered model m8:1 
 [integration_py36_admin_unit_test] 19-07-09:00:28:37 INFO     [clipper_admin.py:642] [admin-l-42930] Done deploying model m8:1. 
 [integration_py36_admin_unit_test] 19-07-09:00:28:37 INFO     [clipper_admin.py:303] [admin-l-42930] Model m8 is now linked to application app7 
 [integration_py36_admin_unit_test] 19-07-09:00:29:07 INFO     [docker_container_manager.py:453] [admin-l-42930] Found 2 replicas for m8:1. Removing 1 
 [integration_py36_admin_unit_test] 19-07-09:00:30:58 INFO     [docker_container_manager.py:453] [admin-l-42930] Found 1 replicas for m8:1. Removing 1 
 [integration_py36_admin_unit_test] ok 
 [integration_py36_admin_unit_test] test_unlinked_app_returns_default_predictions (__main__.ClipperManagerTestCaseLong) ... 19-07-09:00:32:48 INFO     [clipper_admin_tests.py:671] {'query_id': 6, 'output': 'DEFAULT', 'default': True, 'default_explanation': 'No connected models found for query'} 
 [integration_py36_admin_unit_test] ok 
 [integration_py36_admin_unit_test] test_deployed_model_queried_successfully (__main__.ClipperManagerTestCaseLong) ... 19-07-09:00:32:48 INFO     [clipper_admin.py:534] [admin-l-42930] Building model Docker image with model data from /tmp/clipper/tmp320pf9dl 
 [integration_py36_admin_unit_test] 19-07-09:00:32:48 INFO     [clipper_admin.py:539] [admin-l-42930] Step 1/2 : FROM clippertesting/noop-container:c1b57bb7c9 
 [integration_py36_admin_unit_test] 19-07-09:00:32:48 INFO     [clipper_admin.py:539] [admin-l-42930]  ---> 8435f7c6b4ac 
 [integration_py36_admin_unit_test] 19-07-09:00:32:48 INFO     [clipper_admin.py:539] [admin-l-42930] Step 2/2 : COPY /tmp/clipper/tmp320pf9dl /model/ 
 [integration_py36_admin_unit_test] 19-07-09:00:32:48 INFO     [clipper_admin.py:539] [admin-l-42930]  ---> Using cache 
 [integration_py36_admin_unit_test] 19-07-09:00:32:48 INFO     [clipper_admin.py:539] [admin-l-42930]  ---> 88bf956e4b97 
 [integration_py36_admin_unit_test] 19-07-09:00:32:48 INFO     [clipper_admin.py:539] [admin-l-42930] Successfully built 88bf956e4b97 
 [integration_py36_admin_unit_test] 19-07-09:00:32:48 INFO     [clipper_admin.py:539] [admin-l-42930] Successfully tagged admin-l-42930-m5:1 
 [integration_py36_admin_unit_test] 19-07-09:00:32:48 INFO     [clipper_admin.py:541] [admin-l-42930] Pushing model Docker image to admin-l-42930-m5:1 
 [integration_py36_admin_unit_test] 19-07-09:00:32:49 INFO     [docker_container_manager.py:433] [admin-l-42930] Found 0 replicas for m5:1. Adding 1 
 [integration_py36_admin_unit_test] 19-07-09:00:32:50 INFO     [clipper_admin.py:724] [admin-l-42930] Successfully registered model m5:1 
 [integration_py36_admin_unit_test] 19-07-09:00:32:50 INFO     [clipper_admin.py:642] [admin-l-42930] Done deploying model m5:1. 
 [integration_py36_admin_unit_test] 19-07-09:00:32:50 INFO     [clipper_admin.py:303] [admin-l-42930] Model m5 is now linked to application app4 
 [integration_py36_admin_unit_test] 19-07-09:00:33:20 INFO     [clipper_admin_tests.py:693] {'query_id': 7, 'output': 1.0, 'default': False} 
 [integration_py36_admin_unit_test] ok 
 [integration_py36_admin_unit_test] test_batch_queries_returned_successfully (__main__.ClipperManagerTestCaseLong) ... 19-07-09:00:33:20 INFO     [clipper_admin.py:534] [admin-l-42930] Building model Docker image with model data from /tmp/clipper/tmp320pf9dl 
 [integration_py36_admin_unit_test] 19-07-09:00:33:20 INFO     [clipper_admin.py:539] [admin-l-42930] Step 1/2 : FROM clippertesting/noop-container:c1b57bb7c9 
 [integration_py36_admin_unit_test] 19-07-09:00:33:20 INFO     [clipper_admin.py:539] [admin-l-42930]  ---> 8435f7c6b4ac 
 [integration_py36_admin_unit_test] 19-07-09:00:33:20 INFO     [clipper_admin.py:539] [admin-l-42930] Step 2/2 : COPY /tmp/clipper/tmp320pf9dl /model/ 
 [integration_py36_admin_unit_test] 19-07-09:00:33:20 INFO     [clipper_admin.py:539] [admin-l-42930]  ---> Using cache 
 [integration_py36_admin_unit_test] 19-07-09:00:33:20 INFO     [clipper_admin.py:539] [admin-l-42930]  ---> 88bf956e4b97 
 [integration_py36_admin_unit_test] 19-07-09:00:33:20 INFO     [clipper_admin.py:539] [admin-l-42930] Successfully built 88bf956e4b97 
 [integration_py36_admin_unit_test] 19-07-09:00:33:20 INFO     [clipper_admin.py:539] [admin-l-42930] Successfully tagged admin-l-42930-m6:1 
 [integration_py36_admin_unit_test] 19-07-09:00:33:20 INFO     [clipper_admin.py:541] [admin-l-42930] Pushing model Docker image to admin-l-42930-m6:1 
 [integration_py36_admin_unit_test] 19-07-09:00:33:21 INFO     [docker_container_manager.py:433] [admin-l-42930] Found 0 replicas for m6:1. Adding 1 
 [integration_py36_admin_unit_test] 19-07-09:00:33:22 INFO     [clipper_admin.py:724] [admin-l-42930] Successfully registered model m6:1 
 [integration_py36_admin_unit_test] 19-07-09:00:33:22 INFO     [clipper_admin.py:642] [admin-l-42930] Done deploying model m6:1. 
 [integration_py36_admin_unit_test] 19-07-09:00:33:22 INFO     [clipper_admin.py:303] [admin-l-42930] Model m6 is now linked to application app5 
 [integration_py36_admin_unit_test] 19-07-09:00:33:52 INFO     [clipper_admin_tests.py:716] {'batch_predictions': [{'query_id': 8, 'output': 1.0, 'default': False}, {'query_id': 9, 'output': 1.0, 'default': False}, {'query_id': 10, 'output': 1.0, 'default': False}, {'query_id': 11, 'output': 1.0, 'default': False}]} 
 [integration_py36_admin_unit_test] ok 
 [integration_py36_admin_unit_test] test_deployed_python_closure_queried_successfully (__main__.ClipperManagerTestCaseLong) ... 19-07-09:00:33:52 INFO     [deployer_utils.py:41] Saving function to /tmp/tmp5x9iemtaclipper 
 [integration_py36_admin_unit_test] 19-07-09:00:33:52 INFO     [deployer_utils.py:51] Serialized and supplied predict function 
 [integration_py36_admin_unit_test] 19-07-09:00:33:52 INFO     [python.py:192] Python closure saved 
 [integration_py36_admin_unit_test] 19-07-09:00:33:52 INFO     [python.py:206] Using Python 3.6 base image 
 [integration_py36_admin_unit_test] 19-07-09:00:33:52 INFO     [clipper_admin.py:534] [admin-l-42930] Building model Docker image with model data from /tmp/tmp5x9iemtaclipper 
 [integration_py36_admin_unit_test] 19-07-09:00:33:53 INFO     [clipper_admin.py:539] [admin-l-42930] Step 1/2 : FROM clippertesting/python36-closure-container:c1b57bb7c9 
 [integration_py36_admin_unit_test] 19-07-09:00:33:53 INFO     [clipper_admin.py:539] [admin-l-42930]  ---> a1b91d6ce362 
 [integration_py36_admin_unit_test] 19-07-09:00:33:53 INFO     [clipper_admin.py:539] [admin-l-42930] Step 2/2 : COPY /tmp/tmp5x9iemtaclipper /model/ 
 [integration_py36_admin_unit_test] 19-07-09:00:33:53 INFO     [clipper_admin.py:539] [admin-l-42930]  ---> 99375558ba7c 
 [integration_py36_admin_unit_test] 19-07-09:00:33:53 INFO     [clipper_admin.py:539] [admin-l-42930] Successfully built 99375558ba7c 
 [integration_py36_admin_unit_test] 19-07-09:00:33:53 INFO     [clipper_admin.py:539] [admin-l-42930] Successfully tagged admin-l-42930-m4:1 
 [integration_py36_admin_unit_test] 19-07-09:00:33:53 INFO     [clipper_admin.py:541] [admin-l-42930] Pushing model Docker image to admin-l-42930-m4:1 
 [integration_py36_admin_unit_test] 19-07-09:00:33:53 INFO     [docker_container_manager.py:433] [admin-l-42930] Found 0 replicas for m4:1. Adding 1 
 [integration_py36_admin_unit_test] 19-07-09:00:33:54 INFO     [clipper_admin.py:724] [admin-l-42930] Successfully registered model m4:1 
 [integration_py36_admin_unit_test] 19-07-09:00:33:54 INFO     [clipper_admin.py:642] [admin-l-42930] Done deploying model m4:1. 
 [integration_py36_admin_unit_test] 19-07-09:00:33:54 INFO     [clipper_admin.py:303] [admin-l-42930] Model m4 is now linked to application app3 
 [integration_py36_admin_unit_test] ok 
 [integration_py36_admin_unit_test] test_fixed_batch_size_model_processes_specified_query_batch_size_when_saturated (__main__.ClipperManagerTestCaseLong) ... 19-07-09:00:34:55 INFO     [deployer_utils.py:41] Saving function to /tmp/tmp7xdgqyk_clipper 
 [integration_py36_admin_unit_test] 19-07-09:00:34:55 INFO     [deployer_utils.py:51] Serialized and supplied predict function 
 [integration_py36_admin_unit_test] 19-07-09:00:34:55 INFO     [python.py:192] Python closure saved 
 [integration_py36_admin_unit_test] 19-07-09:00:34:55 INFO     [python.py:206] Using Python 3.6 base image 
 [integration_py36_admin_unit_test] 19-07-09:00:34:55 INFO     [clipper_admin.py:534] [admin-l-42930] Building model Docker image with model data from /tmp/tmp7xdgqyk_clipper 
 [integration_py36_admin_unit_test] 19-07-09:00:34:55 INFO     [clipper_admin.py:539] [admin-l-42930] Step 1/2 : FROM clippertesting/python36-closure-container:c1b57bb7c9 
 [integration_py36_admin_unit_test] 19-07-09:00:34:55 INFO     [clipper_admin.py:539] [admin-l-42930]  ---> a1b91d6ce362 
 [integration_py36_admin_unit_test] 19-07-09:00:34:55 INFO     [clipper_admin.py:539] [admin-l-42930] Step 2/2 : COPY /tmp/tmp7xdgqyk_clipper /model/ 
 [integration_py36_admin_unit_test] 19-07-09:00:34:55 INFO     [clipper_admin.py:539] [admin-l-42930]  ---> c321109847e8 
 [integration_py36_admin_unit_test] 19-07-09:00:34:55 INFO     [clipper_admin.py:539] [admin-l-42930] Successfully built c321109847e8 
 [integration_py36_admin_unit_test] 19-07-09:00:34:55 INFO     [clipper_admin.py:539] [admin-l-42930] Successfully tagged admin-l-42930-m7:1 
 [integration_py36_admin_unit_test] 19-07-09:00:34:55 INFO     [clipper_admin.py:541] [admin-l-42930] Pushing model Docker image to admin-l-42930-m7:1 
 [integration_py36_admin_unit_test] 19-07-09:00:34:56 INFO     [docker_container_manager.py:433] [admin-l-42930] Found 0 replicas for m7:1. Adding 1 
 [integration_py36_admin_unit_test] 19-07-09:00:34:57 INFO     [clipper_admin.py:724] [admin-l-42930] Successfully registered model m7:1 
 [integration_py36_admin_unit_test] 19-07-09:00:34:57 INFO     [clipper_admin.py:642] [admin-l-42930] Done deploying model m7:1. 
 [integration_py36_admin_unit_test] 19-07-09:00:34:57 INFO     [clipper_admin.py:303] [admin-l-42930] Model m7 is now linked to application app6 
 [integration_py36_admin_unit_test] ok 
 [integration_py36_admin_unit_test] 19-07-09:00:36:22 INFO     [test_utils.py:107] Creating DockerContainerManager 
 [integration_py36_admin_unit_test] 19-07-09:00:36:22 INFO     [test_utils.py:112] Cleaning up Docker cluster admin-l-42930 
 [integration_py36_admin_unit_test] 19-07-09:00:36:27 INFO     [clipper_admin.py:1424] [admin-l-42930] Stopped all Clipper cluster and all model containers 
 [integration_py36_admin_unit_test]  
 [integration_py36_admin_unit_test] ---------------------------------------------------------------------- 
 [integration_py36_admin_unit_test] Ran 27 tests in 1038.114s 
 [integration_py36_admin_unit_test]  
 [integration_py36_admin_unit_test] OK 
 [integration_py36_admin_unit_test] {'query_id': 12, 'output': 3, 'default': False} 
 [integration_py36_admin_unit_test] 19-07-08:17:36:27 INFO     [retry_with_timeout.py:90] Success! 
===== finished: integration_py36_admin_unit_test =====
+ '[' -z x ']'
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/2107/
Test PASSed.
Finished: SUCCESS