SuccessConsole Output

Skipping 330 KB.. Full Log
gResult(timesteps_total=6105, done=None, info={'sgd_time': 0.5202233791351318, 'sample_throughput': 1972.2297019901694, 'shuffle_time': 0.0002868175506591797, 'kl_divergence': 0.0084442226, 'rollouts_time': 1.7094080448150635, 'load_time': 0.04280400276184082, 'kl_coefficient': 0.2}, episode_reward_mean=22.779850746268657, episode_len_mean=22.779850746268657, episodes_total=None, mean_accuracy=None, mean_validation_accuracy=None, mean_loss=None, neg_mean_loss=None, experiment_id='779b7ff80e9c48c38f3cab0bab0b4805', training_iteration=1, timesteps_this_iter=6105, time_this_iter_s=2.281959056854248, time_total_s=2.281959056854248, pid=1, date='2018-02-02_23-01-05', timestamp=1517612465, hostname='c7cf26fe94ec')
===> iteration 1
Computing policy (iterations=5, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    4.89384e+02   -7.33520e-03    4.89392e+02    1.40229e-04    6.80719e-01
              1    4.89037e+02   -1.33464e-02    4.89051e+02    1.17285e-03    6.75034e-01
              2    4.88620e+02   -1.94726e-02    4.88638e+02    3.76844e-03    6.67250e-01
              3    4.88178e+02   -2.62030e-02    4.88202e+02    8.07215e-03    6.57770e-01
              4    4.87668e+02   -3.16611e-02    4.87697e+02    1.43527e-02    6.46397e-01
current status: TrainingResult(timesteps_total=12184, done=None, info={'sgd_time': 0.12975382804870605, 'sample_throughput': 7837.919044810345, 'shuffle_time': 0.0002079010009765625, 'kl_divergence': 0.014352727, 'rollouts_time': 1.255389928817749, 'load_time': 0.0012011528015136719, 'kl_coefficient': 0.2}, episode_reward_mean=30.857868020304569, episode_len_mean=30.857868020304569, episodes_total=None, mean_accuracy=None, mean_validation_accuracy=None, mean_loss=None, neg_mean_loss=None, experiment_id='779b7ff80e9c48c38f3cab0bab0b4805', training_iteration=2, timesteps_this_iter=6079, time_this_iter_s=1.3950510025024414, time_total_s=3.6770100593566895, pid=1, date='2018-02-02_23-01-06', timestamp=1517612466, hostname='c7cf26fe94ec')
===> iteration 2
Computing policy (iterations=5, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.24547e+03   -2.62900e-02    1.24550e+03    9.07759e-05    6.51467e-01
              1    1.24517e+03   -2.88121e-02    1.24520e+03    7.29088e-04    6.45145e-01
              2    1.24468e+03   -3.16437e-02    1.24471e+03    1.97348e-03    6.39478e-01
              3    1.24415e+03   -3.48622e-02    1.24419e+03    3.76974e-03    6.34193e-01
              4    1.24360e+03   -3.77203e-02    1.24364e+03    6.22783e-03    6.28454e-01
current status: TrainingResult(timesteps_total=18374, done=None, info={'sgd_time': 0.17127442359924316, 'sample_throughput': 5896.969195840067, 'shuffle_time': 0.0003371238708496094, 'kl_divergence': 0.0062278295, 'rollouts_time': 1.5861139297485352, 'load_time': 0.0018870830535888672, 'kl_coefficient': 0.2}, episode_reward_mean=52.905982905982903, episode_len_mean=52.905982905982903, episodes_total=None, mean_accuracy=None, mean_validation_accuracy=None, mean_loss=None, neg_mean_loss=None, experiment_id='779b7ff80e9c48c38f3cab0bab0b4805', training_iteration=3, timesteps_this_iter=6190, time_this_iter_s=1.7691099643707275, time_total_s=5.446120023727417, pid=1, date='2018-02-02_23-01-08', timestamp=1517612468, hostname='c7cf26fe94ec')
Saving checkpoint to object store, 6956119 bytes
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7efc6c389fd0> [ 0.87166013  0.29573842  0.00931639  0.18811973] 0.477 0.4765
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7efc6c389fd0> [ 0.37144903  0.14740338  0.0555936   0.31123588] 0.5445 0.568
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7efc6c38[2018-02-02 23:01:41,621] Making new env: CartPole-v0
Disconnecting client on fd 63
Disconnecting client on fd 80
Disconnecting client on fd 81
Disconnecting client on fd 67
Disconnecting client on fd 62
Disconnecting client on fd 64
Disconnecting client on fd 79
Disconnecting client on fd 68
Disconnecting client on fd 66
Disconnecting client on fd 65
[2018-02-02 23:01:42,197] Making new env: CartPole-v0
[2018-02-02 23:01:44,978] Making new env: CartPole-v0
[2018-02-02 23:01:45,353] Making new env: CartPole-v0
2018-02-02 23:01:45.356098: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
[2018-02-02 23:01:45,382] Making new env: CartPole-v0
[2018-02-02 23:01:45,434] Making new env: CartPole-v0
2018-02-02 23:01:45.655165: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
[2018-02-02 23:01:45,723] Making new env: CartPole-v0
2018-02-02 23:01:45.745072: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
[2018-02-02 23:01:45,753] Making new env: CartPole-v0
[2018-02-02 23:01:45,760] Making new env: CartPole-v0
2018-02-02 23:01:45.812380: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
[2018-02-02 23:01:45,814] Making new env: CartPole-v0
2018-02-02 23:01:46.113837: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-02-02 23:01:46.138313: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-02-02 23:01:46.140106: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-02-02 23:01:46.184976: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
9fd0> [ 0.35956261  0.17569657  0.0669806   0.93213994] 0.6355 0.6345
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7efc6c389fd0> [ 0.57588792  0.6250996   0.14173434  0.08977573] 0.5495 0.553
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7efc6c389fd0> [ 0.70921683  0.2703621   0.28195575  0.92615418] 0.631 0.632
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7efc6c389fd0> [ 0.88233253  0.79102061  0.42830405  0.45982792] 0.5905 0.602
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7efc6c389fd0> [ 0.11064486  0.68364681  0.19823805  0.32358308] 0.7065 0.681
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7efc6c389fd0> [ 0.95585294  0.72704513  0.82432857  0.74047495] 0.6855 0.664
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7efc6c389fd0> [ 0.10357396  0.58798378  0.09070239  0.63604317] 0.673 0.6725
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7efc6c389fd0> [ 0.19099796  0.13695862  0.44116219  0.82176488] 0.794 0.7665
Unified logger created with logdir '/root/ray_results/2018-02-02_23-01-41OpiPru'
Observation shape is (4,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7efe352e9f50>
Setting up loss
Unified logger created with logdir '/root/ray_results/2018-02-02_23-01-42JFStg9'
Observation shape is (4,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7efe352e9f50>
Setting up loss
current status: TrainingResult(timesteps_total=2495, done=None, info={}, episode_reward_mean=25.459183673469386, episode_len_mean=25.459183673469386, episodes_total=None, mean_accuracy=None, mean_validation_accuracy=None, mean_loss=None, neg_mean_loss=None, experiment_id='baf7f1b9c23341af92e385626f4162b8', training_iteration=1, timesteps_this_iter=2495, time_this_iter_s=3.7686259746551514, time_total_s=3.7686259746551514, pid=1, date='2018-02-02_23-01-46', timestamp=1517612506, hostname='c7cf26fe94ec')
current status: TrainingResult(timesteps_total=5109, done=None, info={}, episode_reward_mean=36.305555555555557, episode_len_mean=36.305555555555557, episodes_total=None, mean_accuracy=None, mean_validation_accuracy=None, mean_loss=None, neg_mean_loss=None, experiment_id='baf7f1b9c23341af92e385626f4162b8', training_iteration=2, timesteps_this_iter=2614, time_this_iter_s=0.6014790534973145, time_total_s=4.370105028152466, pid=1, date='2018-02-02_23-01-47', timestamp=1517612507, hostname='c7cf26fe94ec')
current status: TrainingResult(timesteps_total=7825, done=None, info={}, episode_reward_mean=43.806451612903224, episode_len_mean=43.806451612903224, episodes_total=None, mean_accuracy=None, mean_validation_accuracy=None, mean_loss=None, neg_mean_loss=None, experiment_id='baf7f1b9c23341af92e385626f4162b8', training_iteration=3, timesteps_this_iter=2716, time_this_iter_s=0.6936049461364746, time_total_s=5.06370997428894, pid=1, date='2018-02-02_23-01-47', timestamp=1517612507, hostname='c7cf26fe94ec')
Saving checkpoint to object store, 6354644 bytes
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7efcc4259190> [ 0.06899577  0.63940614  0.16706312  0.1743882 ] 0.519 0.5115
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7efcc4259190> [ 0.44068838  0.04181587  0.69309665  0.58992988] 0.6805 0.669
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7efcc4259190> [ 0.3622994   0.72573079  0.92547269  0.22894242] 0.538 0.5275
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7efcc4259190> [ 0.16323198  0.95908973  0.29686541  0.18376958] 0.4685 0.48
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7efcc4259190> [ 0.30748765  0.20853617  0.60852772  0.59679157] 0.668 0.656
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7efcc4259190> [ 0.38726953  0.19883928  0.01561001  0.90958597] 0.694 0.7105
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7efcc4259190> [ 0.95727968  0.37868358  0.20505337  0.3827176 ] 0.5445 0.5615
ChDisconnecting client on fd 11
Disconnecting client on fd 65
Disconnecting client on fd 64
Disconnecting client on fd 63
Disconnecting client on fd 62
Disconnecting client on fd 68
Disconnecting client on fd 67
Disconnecting client on fd 66
Disconnecting client on fd 79
[WARN] (/ray/src/global_scheduler/global_scheduler.cc:412) Missed too many heartbeats from local scheduler, marking as dead.
Disconnecting client on fd 59
Disconnecting client on fd 58
Disconnecting client on fd 57
Disconnecting client on fd 56
Disconnecting client on fd 55
Disconnecting client on fd 54
Disconnecting client on fd 53
Disconnecting client on fd 52
Disconnecting client on fd 51
Disconnecting client on fd 50
Disconnecting client on fd 49
Disconnecting client on fd 48
Disconnecting client on fd 47
Disconnecting client on fd 46
Disconnecting client on fd 45
Disconnecting client on fd 44
Disconnecting client on fd 43
Disconnecting client on fd 42
Disconnecting client on fd 41
Disconnecting client on fd 40
Disconnecting client on fd 39
Disconnecting client on fd 38
Disconnecting client on fd 37
Disconnecting client on fd 36
Disconnecting client on fd 35
Disconnecting client on fd 34
Disconnecting client on fd 33
Disconnecting client on fd 32
Disconnecting client on fd 31
Disconnecting client on fd 30
Disconnecting client on fd 29
Disconnecting client on fd 28
Disconnecting client on fd 27
Disconnecting client on fd 26
Disconnecting client on fd 25
Disconnecting client on fd 24
Disconnecting client on fd 23
Disconnecting client on fd 22
Disconnecting client on fd 21
Disconnecting client on fd 20
Disconnecting client on fd 19
Disconnecting client on fd 18
Disconnecting client on fd 17
Disconnecting client on fd 16
Disconnecting client on fd 15
Disconnecting client on fd 14
Disconnecting client on fd 13
Disconnecting client on fd 12
Disconnecting client on fd 9
Disconnecting client on fd 7
ecking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7efcc4259190> [ 0.0677899   0.74274433  0.77862313  0.84005108] 0.652 0.657
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7efcc4259190> [ 0.48157765  0.55985354  0.38947006  0.21000424] 0.5175 0.5255
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7efcc4259190> [ 0.2531078   0.95029692  0.70769528  0.49079016] 0.5915 0.574
All checkpoint restore tests passed!
+ docker run --rm --shm-size=10G --memory=10G 9cba6202a7e5a1b8b52b05e0fdca82b57fb1f40abc4ec6a4b5dee6a149dcce35 python /ray/python/ray/rllib/test/test_supported_spaces.py
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
testAll (__main__.ModelSupportedSpaces) ... Allowing the Plasma store to use up to 8.58993GB of memory.
Starting object store with directory /dev/shm and huge page support disabled
2018-02-02 23:02:27.919246: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:02:28,012] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:02:28,146] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:02:29,108] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:02:29,218] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:02:29,978] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:02:30,256] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:02:31,147] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:02:31,267] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:02:32,207] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:02:32,331] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:02:33,216] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:02:33,310] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:02:33,989] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:02:34,111] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
Waiting for redis server at 127.0.0.1:56497 to respond...
Waiting for redis server at 127.0.0.1:29250 to respond...
Warning: Reducing object store memory because /dev/shm has only 10737418240 bytes available. You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you may need to pass an argument with the flag '--shm-size' to 'docker run'.
Starting local scheduler with the following resources: {'GPU': 0, 'CPU': 48}.

======================================================================
View the web UI at http://localhost:8888/notebooks/ray_ui66866.ipynb?token=9b5ff0c021641f5ab3807b06e6d98b24886b442d75ebd7b4
======================================================================

('=== Testing', 'DQN', Discrete(5), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-27pqXZlz'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
ok
()
('=== Testing', 'DQN', Discrete(5), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-29HRpm6M'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
ok
()
('=== Testing', 'DQN', Discrete(5), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-29Ggwstn'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
ok
()
('=== Testing', 'DQN', Discrete(5), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-31EG5Lie'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
ok
()
('=== Testing', 'DQN', Discrete(5), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-32X63QYd'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
ok
()
('=== Testing', 'DQN', Discrete(5), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-32vbCNDC'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
ok
()
('=== Testing', 'DQN', Discrete(5), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-3340P63y'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
ok
()
('=== Testing', 'DQN', Box(5,), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-341npaY0'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', Box(5,), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-34l492Qo'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
unsupported
()
('=== Testing', 'DQN', Box(5,), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-34Lzx1RJ'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', Box(5,), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-344mS9OZ'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
unsupported
()
('=== Testing', 'DQN', Box(5,), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-34RbpAL8'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', Box(5,), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-34RnzXk_'
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', Box(5,), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-34nkQes3'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
unsupported
()
('=== Testing', 'DQN', [Box(5,), Box(5,)], Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-34K4WkKJ'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', [Box(5,), Box(5,)], Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-34ipIqJN'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
unsupported
()
('=== Testing', 'DQN', [Box(5,), Box(5,)], Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-34ESTyVj'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', [Box(5,), Box(5,)], Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-34SVXVLb'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
unsupported
()
('=== Testing', 'DQN', [Box(5,), Box(5,)], Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-34efdf8E'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', [Box(5,), Box(5,)], Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-348z6UY3'
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', [Box(5,), Box(5,)], Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-34pmM767'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
unsupported
()
('=== Testing', 'DQN', Tuple(Box(5,), Box(5,)), Tuple(Discrete(10), Box(5,)), '===2018-02-02 23:02:44.511348: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 62
2018-02-02 23:02:51.534346: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 64
2018-02-02 23:02:54.860608: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 65
2018-02-02 23:02:58.240083: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 65
2018-02-02 23:03:01.716006: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-34Z0ss1A'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', Tuple(Box(5,), Box(5,)), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-34Z35zPe'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
unsupported
()
('=== Testing', 'DQN', Tuple(Box(5,), Box(5,)), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-34r62CHc'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', Tuple(Box(5,), Box(5,)), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-34p4GvMY'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
unsupported
()
('=== Testing', 'DQN', Tuple(Box(5,), Box(5,)), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-342P25tE'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', Tuple(Box(5,), Box(5,)), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-34_ar0_4'
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', Tuple(Box(5,), Box(5,)), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-345i3WRc'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
unsupported
()
('=== Testing', 'A3C', Discrete(5), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-35Ex8nLY'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
WARNING: Serializing objects of type <class 'ray.tune.registry._Registry'> by expanding them as dictionaries of their fields. This behavior may be incorrect in some cases.
ok
()
('=== Testing', 'A3C', Discrete(5), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-447OklHR'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
ok
()
('=== Testing', 'A3C', Discrete(5), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-51_tXY2d'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
Setting up loss
ok
()
('=== Testing', 'A3C', Discrete(5), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-55rNEiYj'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
ok
()
('=== Testing', 'A3C', Discrete(5), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-02-586aYgJx'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
ok
()
(Disconnecting client on fd 65
2018-02-02 23:03:05.018145: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 65
2018-02-02 23:03:08.376394: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:03:11.824907: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:03:15.192853: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:03:18.878845: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:03:22.305657: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:03:25.791140: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:03:29.321533: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:03:32.824324: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
'=== Testing', 'A3C', Discrete(5), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-01QxHioK'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
ok
()
('=== Testing', 'A3C', Discrete(5), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-05Xz4Nkr'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
Setting up loss
ok
()
('=== Testing', 'A3C', Box(5,), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-08OA0ng5'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
ok
()
('=== Testing', 'A3C', Box(5,), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-12RhBhQ6'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
ok
()
('=== Testing', 'A3C', Box(5,), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-15xYArR9'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
Setting up loss
ok
()
('=== Testing', 'A3C', Box(5,), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-19nuIGqt'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
ok
()
('=== Testing', 'A3C', Box(5,), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-22hHSGF1'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
ok
()
('=== Testing', 'A3C', Box(5,), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-252fnHs0'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
ok
()
('=== Testing', 'A3C', Box(5,), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-2950hLF0'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
Setting up loss
ok
()
('=== Testing', 'A3C', [Box(5,), Box(5,)], Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-33OxlL62'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', [Box(5,), Box(5,)], Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-33aOzN6x'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', [Box(5,), Box(5,)], Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-33Jw_IIE'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', [Box(5,), Box(5,)], Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-33jIFs1b'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', [Box(5,), Box(5,)], Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-335J1_yl'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', [Box(5,), Box(5,)], Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-33VWKyaO'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', [Box(5,), Box(5,)], Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-33l3oa8a'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', Tuple(Box(5,), Box(5,)), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-34bJASmI'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', Tuple(Box(5,), Box(5,)), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-34QEtVOP'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', Tuple(Box(5,), Box(5,)), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-34OYAR8i'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', Tuple(Box(5,), Box(5,)), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-34fNK9Ty'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', Tuple(Box(5,), Box(5,)), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-34dY1gwh'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', Tuple(Box(5,), Box(5,)), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-34E3_5EG'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', Tuple(Box(5,), Box(5,)), Box(210, 160, 3), '===')
Unified logger created with logdir2018-02-02 23:03:37.887378: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:03:42.218699: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:03:46.823313: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
 '/root/ray_results/2018-02-02_23-03-34eFFF9R'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'PPO', Discrete(5), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-34lmMeEQ'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.00236e+00    0.00000e+00    1.00236e+00    0.00000e+00    1.60944e+00
ok
()
('=== Testing', 'PPO', Discrete(5), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-38BSozbE'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.86623e-01    0.00000e+00    9.86623e-01    0.00000e+00    1.60944e+00
ok
()
('=== Testing', 'PPO', Discrete(5), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-43RfarDo'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    8.18109e-01    0.00000e+00    8.18109e-01    0.00000e+00    1.60904e+00
ok
()
('=== Testing', 'PPO', Discrete(5), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-485jwTtS'
Observation shape is (128,)
Assuming Atari ram enDisconnecting client on fd 70
2018-02-02 23:03:51.700709: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:03:56.020405: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:04:00.496976: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:04:04.857920: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:04:09.645684: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
v, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.96703e-01    0.00000e+00    9.96703e-01    0.00000e+00    1.60943e+00
ok
()
('=== Testing', 'PPO', Discrete(5), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-525Rj3Vf'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.94247e-01    0.00000e+00    9.94247e-01    0.00000e+00    1.60943e+00
ok
()
('=== Testing', 'PPO', Discrete(5), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-03-57AeDxsy'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.90864e-01    0.00000e+00    9.90864e-01    0.00000e+00    1.60943e+00
ok
()
('=== Testing', 'PPO', Discrete(5), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-04-01qJPjRD'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.96071e-01    0.00000e+00    9.96071e-01   -2.28859e-08    1.60742e+00
ok
()
('=== Testing', 'PPO', Box(5,), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-04-06aTJAeY'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.97299e-01    0.00000e+00    9.97299e-01    0.00000e+00    7.09311e+00
ok
()
('=== Testing', 'PPO', Box(5,), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-04-10Lx1L9B'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> itera2018-02-02 23:04:13.898507: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:04:18.358407: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:04:23.216724: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:04:27.609033: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:04:31.980340: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:04:36.735396: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
tion 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.89242e-01    0.00000e+00    9.89242e-01    0.00000e+00    7.09419e+00
ok
()
('=== Testing', 'PPO', Box(5,), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-04-14C2sAlq'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.04202e+00    0.00000e+00    1.04202e+00    0.00000e+00    7.00752e+00
ok
()
('=== Testing', 'PPO', Box(5,), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-04-19Z3XFSp'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.97509e-01    0.00000e+00    9.97509e-01    0.00000e+00    7.09322e+00
ok
()
('=== Testing', 'PPO', Box(5,), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-04-244xXJag'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.00370e+00    0.00000e+00    1.00370e+00    0.00000e+00    7.10117e+00
ok
()
('=== Testing', 'PPO', Box(5,), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-04-28hyWY3g'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.90462e-01    0.00000e+00    9.90462e-01    0.00000e+00    7.07905e+00
ok
()
('=== Testing', 'PPO', Box(5,), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-04-33RnF2uq'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.20165e+00    0.00000e+00    1.20165e+00    0.00000e+00    7.40707e+00
ok
()
('=== Testing', 'PPO', [Box(5,), Box(5,)], Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-04-38UzCM3P'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <2018-02-02 23:04:41.860495: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:04:47.105816: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:04:52.281503: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:04:57.648210: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:05:02.738313: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:05:07.293852: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.98126e-01    0.00000e+00    9.98126e-01    0.00000e+00    1.41720e+01
ok
()
('=== Testing', 'PPO', [Box(5,), Box(5,)], Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-04-43lqr4Xy'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.92037e-01    0.00000e+00    9.92037e-01    0.00000e+00    1.41806e+01
ok
()
('=== Testing', 'PPO', [Box(5,), Box(5,)], Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-04-48CO8VVp'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.04948e+00    0.00000e+00    1.04948e+00    0.00000e+00    1.42047e+01
ok
()
('=== Testing', 'PPO', [Box(5,), Box(5,)], Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-04-54PyGgYk'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.94674e-01    0.00000e+00    9.94674e-01    0.00000e+00    1.42033e+01
ok
()
('=== Testing', 'PPO', [Box(5,), Box(5,)], Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-04-59rEivVW'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.94916e-01    0.00000e+00    9.94916e-01    0.00000e+00    1.41700e+01
ok
()
('=== Testing', 'PPO', [Box(5,), Box(5,)], Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-05-046jcH8r'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.01052e+00    0.00000e+00    1.01052e+00    0.00000e+00    1.41757e+01
ok
()
('=== Testing', 'PPO', [Box(5,), Box(5,)], Box(210, 160, 3), '===')
Unified logger createDisconnecting client on fd 71
2018-02-02 23:05:12.291604: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:05:17.715170: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:05:22.586210: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:05:27.878422: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:05:33.301982: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:05:38.654317: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
d with logdir '/root/ray_results/2018-02-02_23-05-08Bw4iWi'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.76706e-01    0.00000e+00    9.76706e-01    0.00000e+00    1.40994e+01
ok
()
('=== Testing', 'PPO', Tuple(Box(5,), Box(5,)), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-05-13Hsphk2'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.00708e+00    0.00000e+00    1.00708e+00    0.00000e+00    1.41998e+01
ok
()
('=== Testing', 'PPO', Tuple(Box(5,), Box(5,)), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-05-19JgjEHN'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.00459e+00    0.00000e+00    1.00459e+00    0.00000e+00    1.42043e+01
ok
()
('=== Testing', 'PPO', Tuple(Box(5,), Box(5,)), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-05-24FpUWfo'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.29885e-01    0.00000e+00    9.29885e-01    0.00000e+00    1.41312e+01
ok
()
('=== Testing', 'PPO', Tuple(Box(5,), Box(5,)), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-05-29_GJ_Bp'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.94723e-01    0.00000e+00    9.94723e-01    0.00000e+00    1.42167e+01
ok
()
('=== Testing', 'PPO', Tuple(Box(5,), Box(5,)), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-05-34K0r1El'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter Disconnecting client on fd 71
2018-02-02 23:05:44.239975: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:05:49.542728: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:05:54.307398: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/es/es.py:262: RuntimeWarning: Mean of empty slice.
  tlogger.record_tabular("EvalEpRewMean", eval_returns.mean())
/opt/conda/lib/python2.7/site-packages/numpy/core/_methods.py:80: RuntimeWarning: invalid value encountered in double_scalars
  ret = ret.dtype.type(ret / rcount)
/opt/conda/lib/python2.7/site-packages/numpy/core/_methods.py:135: RuntimeWarning: Degrees of freedom <= 0 for slice
  keepdims=keepdims)
/opt/conda/lib/python2.7/site-packages/numpy/core/_methods.py:105: RuntimeWarning: invalid value encountered in true_divide
  arrmean, rcount, out=arrmean, casting='unsafe', subok=False)
/opt/conda/lib/python2.7/site-packages/numpy/core/_methods.py:127: RuntimeWarning: invalid value encountered in double_scalars
  ret = ret.dtype.type(ret / rcount)
/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/es/es.py:264: RuntimeWarning: Mean of empty slice.
  tlogger.record_tabular("EvalEpLenMean", eval_lengths.mean())
    total loss    policy loss        vf loss             kl        entropy
              0    1.00009e+00    0.00000e+00    1.00009e+00    0.00000e+00    1.41728e+01
ok
()
('=== Testing', 'PPO', Tuple(Box(5,), Box(5,)), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-05-40ohQr1t'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.92656e-01    0.00000e+00    9.92656e-01    0.00000e+00    1.41931e+01
ok
()
('=== Testing', 'PPO', Tuple(Box(5,), Box(5,)), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-05-452nXTAu'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.06530e+00    0.00000e+00    1.06530e+00    0.00000e+00    1.42342e+01
ok
()
('=== Testing', 'ES', Discrete(5), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-05-5160UxA5'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 517      |
| GradNorm            | 91.1     |
| UpdateRatio         | 0.118    |
| EpisodesThisIter    | 8        |
| EpisodesSoFar       | 8        |
| TimestepsThisIter   | 8        |
| TimestepsSoFar      | 8        |
| TimeElapsedThisIter | 2.69     |
| TimeElapsed         | 2.69     |
----------------------------------
/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/es/es.py:296: RuntimeWarning: Mean of empty slice.
  episode_reward_mean=eval_returns.mean(),
/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/es/es.py:297: RuntimeWarning: Mean of empty slice.
  episode_len_mean=eval_lengths.mean(),
Disconnecting client on fd 71
2018-02-02 23:05:57.631728: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Discrete(5), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-05-54eKXxYN'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 42       |
| UpdateRatio         | 0.116    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 2.59     |
| TimeElapsed         | 2.59     |
----------------------------------
Disconnecting client on fd 72
2018-02-02 23:06:01.072204: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Discrete(5), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-05-57JMWFmP'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 258      |
| GradNorm            | 5.5e+03  |
| UpdateRatio         | 1.33     |
| EpisodesThisIter    | 6        |
| EpisodesSoFar       | 6        |
| TimestepsThisIter   | 6        |
| TimestepsSoFar      | 6        |
| TimeElapsedThisIter | 2.9      |
| TimeElapsed         | 2.91     |
----------------------------------
Disconnecting client on fd 72
2018-02-02 23:06:04.681281: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Discrete(5), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-01d8i3YM'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 521      |
| GradNorm            | 127      |
| UpdateRatio         | 0.14     |
| EpisodesThisIter    | 8        |
| EpisodesSoFar       | 8        |
| TimestepsThisIter   | 8        |
| TimestepsSoFar      | 8        |
| TimeElapsedThisIter | 2.6      |
| TimeElapsed         | 2.6      |
----------------------------------
Disconnecting client on fd 72
2018-02-02 23:06:07.870279: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Discrete(5), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-05ioqX1P'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 43.5     |
| UpdateRatio         | 0.117    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 2.6      |
| TimeElapsed         | 2.6      |
----------------------------------
Disconnecting client on fd 72
2018-02-02 23:06:11.196664: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Discrete(5), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-083DzqET'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 42.4     |
| UpdateRatio         | 0.116    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 2.74     |
| TimeElapsed         | 2.74     |
----------------------------------
Disconnecting client on fd 72
2018-02-02 23:06:14.582964: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Discrete(5), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-11IXkty3'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 262      |
| GradNorm            | 2.3e+04  |
| UpdateRatio         | 1.31     |
| EpisodesThisIter    | 4        |
| EpisodesSoFar       | 4        |
| TimestepsThisIter   | 4        |
| TimestepsSoFar      | 4        |
| TimeElapsedThisIter | 2.74     |
| TimeElapsed         | 2.74     |
----------------------------------
Disconnecting client on fd 72
2018-02-02 23:06:18.051027: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Box(5,), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-15SzulA3'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 517      |
| GradNorm            | 44.1     |
| UpdateRatio         | 0.118    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 2.56     |
| TimeElapsed         | 2.56     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:06:21.359818: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Box(5,), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-18ZNLq1M'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 517      |
| GradNorm            | 87.4     |
| UpdateRatio         | 0.116    |
| EpisodesThisIter    | 8        |
| EpisodesSoFar       | 8        |
| TimestepsThisIter   | 8        |
| TimestepsSoFar      | 8        |
| TimeElapsedThisIter | 2.53     |
| TimeElapsed         | 2.53     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:06:24.710812: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Box(5,), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-2123jlDB'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 258      |
| GradNorm            | 5.51e+03 |
| UpdateRatio         | 1.33     |
| EpisodesThisIter    | 6        |
| EpisodesSoFar       | 6        |
| TimestepsThisIter   | 6        |
| TimestepsSoFar      | 6        |
| TimeElapsedThisIter | 2.8      |
| TimeElapsed         | 2.8      |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:06:28.274252: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Box(5,), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-25aq3uxG'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 520      |
| GradNorm            | 34.1     |
| UpdateRatio         | 0.14     |
| EpisodesThisIter    | 12       |
| EpisodesSoFar       | 12       |
| TimestepsThisIter   | 12       |
| TimestepsSoFar      | 12       |
| TimeElapsedThisIter | 2.59     |
| TimeElapsed         | 2.59     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:06:31.572261: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Box(5,), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-28MpCwgw'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 516      |
| GradNorm            | 23.9     |
| UpdateRatio         | 0.117    |
| EpisodesThisIter    | 12       |
| EpisodesSoFar       | 12       |
| TimestepsThisIter   | 12       |
| TimestepsSoFar      | 12       |
| TimeElapsedThisIter | 2.54     |
| TimeElapsed         | 2.54     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:06:34.862110: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Box(5,), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-312p4Q9O'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 87.1     |
| UpdateRatio         | 0.116    |
| EpisodesThisIter    | 8        |
| EpisodesSoFar       | 8        |
| TimestepsThisIter   | 8        |
| TimestepsSoFar      | 8        |
| TimeElapsedThisIter | 2.6      |
| TimeElapsed         | 2.6      |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:06:38.236824: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Box(5,), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-35MnObFI'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 262      |
| GradNorm            | 2.3e+04  |
| UpdateRatio         | 1.31     |
| EpisodesThisIter    | 4        |
| EpisodesSoFar       | 4        |
| TimestepsThisIter   | 4        |
| TimestepsSoFar      | 4        |
| TimeElapsedThisIter | 2.77     |
| TimeElapsed         | 2.77     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:06:41.915128: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', [Box(5,), Box(5,)], Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-38xoUs89'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 46.8     |
| UpdateRatio         | 0.121    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 2.44     |
| TimeElapsed         | 2.44     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:06:45.396668: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', [Box(5,), Box(5,)], Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-421pAWFh'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 516      |
| GradNorm            | 45       |
| UpdateRatio         | 0.119    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 2.74     |
| TimeElapsed         | 2.74     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:06:48.994221: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', [Box(5,), Box(5,)], Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-4569MJ1B'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 288      |
| GradNorm            | 5.53e+03 |
| UpdateRatio         | 1.17     |
| EpisodesThisIter    | 6        |
| EpisodesSoFar       | 6        |
| TimestepsThisIter   | 6        |
| TimestepsSoFar      | 6        |
| TimeElapsedThisIter | 2.92     |
| TimeElapsed         | 2.92     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:06:52.730011: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', [Box(5,), Box(5,)], Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-49fjW0CY'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 521      |
| GradNorm            | 64.1     |
| UpdateRatio         | 0.142    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 2.82     |
| TimeElapsed         | 2.82     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:06:56.032619: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', [Box(5,), Box(5,)], Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-53vHSVB0'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 93.9     |
| UpdateRatio         | 0.12     |
| EpisodesThisIter    | 8        |
| EpisodesSoFar       | 8        |
| TimestepsThisIter   | 8        |
| TimestepsSoFar      | 8        |
| TimeElapsedThisIter | 2.57     |
| TimeElapsed         | 2.57     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:06:59.386756: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', [Box(5,), Box(5,)], Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-567kVbiY'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 519      |
| GradNorm            | 92.7     |
| UpdateRatio         | 0.119    |
| EpisodesThisIter    | 8        |
| EpisodesSoFar       | 8        |
| TimestepsThisIter   | 8        |
| TimestepsSoFar      | 8        |
| TimeElapsedThisIter | 2.61     |
| TimeElapsed         | 2.61     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:07:03.089728: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', [Box(5,), Box(5,)], Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-06-59NxxQkZ'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 292      |
| GradNorm            | 2.3e+04  |
| UpdateRatio         | 1.15     |
| EpisodesThisIter    | 4        |
| EpisodesSoFar       | 4        |
| TimestepsThisIter   | 4        |
| TimestepsSoFar      | 4        |
| TimeElapsedThisIter | 3.04     |
| TimeElapsed         | 3.04     |
----------------------------------
Disconnecting client on fd 74
2018-02-02 23:07:06.677192: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Tuple(Box(5,), Box(5,)), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-07-03aFNLko'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 46.5     |
| UpdateRatio         | 0.121    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 2.61     |
| TimeElapsed         | 2.61     |
----------------------------------
Disconnecting client on fd 74
2018-02-02 23:07:09.754311: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Tuple(Box(5,), Box(5,)), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-07-07f0lhdR'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 44.8     |
| UpdateRatio         | 0.119    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 2.37     |
| TimeElapsed         | 2.37     |
----------------------------------
Disconnecting client on fd 74
2018-02-02 23:07:13.149603: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Tuple(Box(5,), Box(5,)), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-07-10QAqzNn'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 287      |
| GradNorm            | 5.53e+03 |
| UpdateRatio         | 1.17     |
| EpisodesThisIter    | 6        |
| EpisodesSoFar       | 6        |
| TimestepsThisIter   | 6        |
| TimestepsSoFar      | 6        |
| TimeElapsedThisIter | 2.83     |
| TimeElapsed         | 2.83     |
----------------------------------
Disconnecting client on fd 74
2018-02-02 23:07:16.647568: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Tuple(Box(5,), Box(5,)), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-07-13gmjJiw'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 521      |
| GradNorm            | 64.3     |
| UpdateRatio         | 0.142    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 2.65     |
| TimeElapsed         | 2.65     |
----------------------------------
Disconnecting client on fd 74
2018-02-02 23:07:19.822812: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Tuple(Box(5,), Box(5,)), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-07-16g4bXNo'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 45.6     |
| UpdateRatio         | 0.12     |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 2.56     |
| TimeElapsed         | 2.56     |
----------------------------------
Disconnecting client on fd 74
2018-02-02 23:07:23.182590: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Tuple(Box(5,), Box(5,)), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-07-20Lo6slC'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f14386cab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 44.6     |
| UpdateRatio         | 0.119    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 2.63     |
| TimeElapsed         | 2.63     |
----------------------------------
Disconnecting client on fd 74
2018-02-02 23:07:26.619599: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Tuple(Box(5,), Box(5,)), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-07-23RdPtC1'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 291      |
| GradNorm            | 2.31e+04 |
| UpdateRatio         | 1.15     |
| EpisodesThisIter    | 4        |
| EpisodesSoFar       | 4        |
| TimestepsThisIter   | 4        |
| TimestepsSoFar      | 4        |
| TimeElapsedThisIter | 2.76     |
| TimeElapsed         | 2.76     |
----------------------------------
ok
()
('A3C', 'action_space', 'discrete', 'obs_space', 'atari', 'result', 'ok')
('A3C', 'action_space', 'discrete', 'obs_space', 'atari_ram', 'result', 'ok')
('A3C', 'action_space', 'discrete', 'obs_space', 'discrete', 'result', 'ok')
('A3C', 'action_space', 'discrete', 'obs_space', 'image', 'result', 'ok')
('A3C', 'action_space', 'discrete', 'obs_space', 'mixed_tuple', 'result', 'ok')
('A3C', 'action_space', 'discrete', 'obs_space', 'simple_tuple', 'result', 'ok')
('A3C', 'action_space', 'discrete', 'obs_space', 'vector', 'result', 'ok')
('A3C', 'action_space', 'implicit_tuple', 'obs_space', 'atari', 'result', 'ERROR')
('A3C', 'action_space', 'implicit_tuple', 'obs_space', 'atari_ram', 'result', 'ERROR')
('A3C', 'action_space', 'implicit_tuple', 'obs_space', 'discrete', 'result', 'ERROR')
('A3C', 'action_space', 'implicit_tuple', 'obs_space', 'image', 'result', 'ERROR')
('A3C', 'action_space', 'implicit_tuple', 'obs_space', 'mixed_tuple', 'result', 'ERROR')
('A3C', 'action_space', 'implicit_tuple', 'obs_space', 'simple_tuple', 'result', 'ERROR')
('A3C', 'action_space', 'implicit_tuple', 'obs_space', 'vector', 'result', 'ERROR')
('A3C', 'action_space', 'simple_tuple', 'obs_space', 'atari', 'result', 'ERROR')
('A3C', 'action_space', 'simple_tuple', 'obs_space', 'atari_ram', 'result', 'ERROR')
('A3C', 'action_space', 'simple_tuple', 'obs_space', 'discrete', 'result', 'ERROR')
('A3C', 'action_space', 'simple_tuple', 'obs_space', 'image', 'result', 'ERROR')
('A3C', 'action_space', 'simple_tuple', 'obs_space', 'mixed_tuple', 'result', 'ERROR')
('A3C', 'action_space', 'simple_tuple', 'obs_space', 'simple_tuple', 'result', 'ERROR')
('A3C', 'action_space', 'simple_tuple', 'obs_space', 'vector', 'result', 'ERROR')
('A3C', 'action_space', 'vector', 'obs_space', 'atari', 'result', 'ok')
('A3C', 'action_space', 'vector', 'obs_space', 'atari_ram', 'result', 'ok')
('A3C', 'action_space', 'vector', 'obs_space', 'discrete', 'result', 'ok')
('A3C', 'action_space', 'vector', 'obs_space', 'image', 'result', 'ok')
('A3C', 'action_space', 'vector', 'obs_space', 'mixed_tuple', 'result', 'ok')
('A3C', 'action_space', 'vector', 'obs_space', 'simple_tuple', 'result', 'ok')
('A3C', 'action_space', 'vector', 'obs_space', 'vector', 'result', 'ok')
('DQN', 'action_space', 'discrete', 'obs_space', 'atari', 'result', 'ok')
('DQN', 'action_space', 'discrete', 'obs_space', 'atari_ram', 'result', 'ok')
('DQN', 'action_space', 'discrete', 'obs_space', 'discrete', 'result', 'ok')
('DQN', 'action_space', 'discrete', 'obs_space', 'image', 'result', 'ok')
('DQN', 'action_space', 'discrete', 'obs_space', 'mixed_tuple', 'result', 'ok')
('DQN', 'action_space', 'discrete', 'obs_space', 'simple_tuple', 'result', 'ok')
('DQN', 'action_space', 'discrete', 'obs_space', 'vector', 'result', 'ok')
('DQN', 'action_space', 'implicit_tuple', 'obs_space', 'atari', 'result', 'unsupported')
('DQN', 'action_space', 'implicit_tuple', 'obs_space', 'atari_ram', 'result', 'unsupported')
('DQN', 'action_space', 'implicit_tuple', 'obs_space', 'discrete', 'result', 'unsupported')
('DQN', 'action_space', 'implicit_tuple', 'obs_space', 'image', 'result', 'unsupported')
('DQN', 'action_space', 'implicit_tuple', 'obs_space', 'mixed_tuple', 'result', 'unsupported')
('DQN', 'action_space', 'implicit_tuple', 'obs_space', 'simple_tuple', 'result', 'unsupported')
('DQN', 'action_space', 'implicit_tuple', 'obs_space', 'vector', 'result', 'unsupported')
('DQN', 'action_space', 'simple_tuple', 'obs_space', 'atari', 'result', 'unsupported')
('DQN', 'action_space', 'simple_tuple', 'obs_space', 'atari_ram', 'result', 'unsupported')
('DQN', 'action_space', 'simple_tuple', 'obs_space', 'discrete', 'result', 'unsupported')
('DQN', 'action_space', 'simple_tuple', 'obs_space', 'image', 'result', 'unsupported')
('DQN', 'action_space', 'simple_tuple', 'obs_space', 'mixed_tuple', 'result', 'unsupported')
('DQN', 'action_space', 'simple_tuple', 'obs_space', 'simple_tuple', 'result', 'unsupported')
('DQN', 'action_space', 'simple_tuple', 'obs_space', 'vector', 'result', 'unsupported')
('DQN', 'action_space', 'vector', 'obs_space', 'atari', 'result', 'unsupported')
('DQN', 'action_space', 'vector', 'obs_space', 'atari_ram', 'result', 'unsupported')
('DQN', 'action_space', 'vector', 'obs_space', 'discrete', 'result', 'unsupported')
('DQN', 'action_space', 'vector', 'obs_space', 'image', 'result', 'unsupported')
('DQN', 'action_space', 'vector', 'obs_space', 'mixed_tuple', 'result', 'unsupported')
('DQN', 'action_space', 'vector', 'obs_space', 'simple_tuple', 'result', 'unsupported')
('DQN', 'action_space', 'vector', 'obs_space', 'vector', 'result', 'unsupported')
('ES', 'action_space', 'discrete', 'obs_space', 'atari', 'result', 'ok')
('ES', 'action_space', 'discrete', 'obs_space', 'atari_ram', 'result', 'ok')
('ES', 'action_space', 'discrete', 'obs_space', 'discrete', 'result', 'ok')
('ES', 'action_space', 'discrete', 'obs_space', 'image', 'result', 'ok')
('ES', 'action_space', 'discrete', 'obs_space', 'mixed_tuple', 'result', 'ok')
('ES', 'action_space', 'discrete', 'obs_space', 'simple_tuple', 'result', 'ok')
('ES', 'action_space', 'discrete', 'obs_space', 'vector', 'result', 'ok')
('ES', 'action_space', 'implicit_tuple', 'obs_space', 'atari', 'result', 'ok')
('ES', 'action_space', 'implicit_tuple', 'obs_space', 'atari_ram', 'result', 'ok')
('ES', 'action_space', 'implicit_tuple', 'obs_space', 'discrete', 'result', 'ok')
('ES', 'action_space', 'implicit_tuple', 'obs_space', 'image', 'result', 'ok')
('ES', 'action_space', 'implicit_tuple', 'obs_space', 'mixed_tuple', 'result', 'ok')
('ES', 'action_space', 'implicit_tuple', 'obs_space', 'simple_tuple', 'result', 'ok')
('ES', 'action_space', 'implicit_tuple', 'obs_space', 'vector', 'result', 'ok')
('ES', 'action_space', 'simple_tuple', 'obs_space', 'atari', 'result', 'ok')
('ES', 'action_space', 'simple_tuple', 'obs_space', 'atari_ram', 'result', 'ok')
('ES', 'action_space', 'simple_tuple', 'obs_space', 'discrete', 'result', 'ok')
('ES', 'action_space', 'simple_tuple', 'obs_space', 'image', 'result', 'ok')
('ES', 'action_space', 'simple_tuple', 'obs_space', 'mixed_tuple', 'result', 'ok')
('ES', 'action_space', 'simple_tuple', 'obs_space', 'simple_tuple', 'result', 'ok')
('ES', 'action_space', 'simple_tuple', 'obs_space', 'vector', 'result', 'ok')
('ES', 'action_space', 'vector', 'obs_space', 'atari', 'result', 'ok')
('ES', 'action_space', 'vector', 'obs_space', 'atari_ram', 'result', 'ok')
('ES', 'action_space', 'vector', 'obs_space', 'discrete', 'result', 'ok')
('ES', 'action_space', 'vector', 'obs_space', 'image', 'result', 'ok')
('ES', 'action_space', 'vector', 'obs_space', 'mixed_tuple', 'result', 'ok')
('ES', 'action_space', 'vector', 'obs_space', 'simple_tuple', 'result', 'ok')
('ES', 'action_space', 'vector', 'obs_space', 'vector', 'result', 'ok')
('PPO', 'action_space', 'discrete', 'obs_space', 'atari', 'result', 'ok')
('PPO', 'action_space', 'discrete', 'obs_space', 'atari_ram', 'result', 'ok')
('PPO', 'action_space', 'discrete', 'obs_space', 'discrete', 'result', 'ok')
('PPO', 'action_space', 'discrete', 'obs_space', 'image', 'result', 'ok')
('PPO', 'action_space', 'discrete', 'obs_space', 'mixed_tuple', 'result', 'ok')
('PPO', 'action_space', 'discrete', 'obs_space', 'simple_tuple', 'result', 'ok')
('PPO', 'action_space', 'discrete', 'obs_space', 'vector', 'result', 'ok')
('PPO', 'action_space', 'implicit_tuple', 'obs_space', 'atari', 'result', 'ok')
('PPO', 'action_space', 'implicit_tuple', 'obs_space', 'atari_ram', 'result', 'ok')
('PPO', 'action_space', 'implicit_tuple', 'obs_space', 'discrete', 'result', 'ok')
('PPO', 'action_space', 'implicit_tuple', 'obs_space', 'image', 'result', 'ok')
('PPO', 'action_space', 'implicit_tuple', 'obs_space', 'mixed_tuple', 'result', 'ok')
('PPO', 'action_space', 'implicit_tuple', 'obs_space', 'simple_tuple', 'result', 'ok')
('PPO', 'action_space', 'implicit_tuple', 'obs_space', 'vector', 'result', 'ok')
('PPO', 'action_space', 'simple_tuple', 'obs_space', 'atari', 'result', 'ok')
('PPO', 'action_space', 'simple_tuple', 'obs_space', 'atari_ram', 'result', 'ok')
('PPO', 'action_space', 'simple_tuple', 'obs_space', 'discrete', 'rok

----------------------------------------------------------------------
Ran 1 test in 300.368s

OK
esult', 'ok')
('PPO', 'action_space', 'simple_tuple', 'obs_space', 'image', 'result', 'ok')
('PPO', 'action_space', 'simple_tuple', 'obs_space', 'mixed_tuple', 'result', 'ok')
('PPO', 'action_space', 'simple_tuple', 'obs_space', 'simple_tuple', 'result', 'ok')
('PPO', 'action_space', 'simple_tuple', 'obs_space', 'vector', 'result', 'ok')
('PPO', 'action_space', 'vector', 'obs_space', 'atari', 'result', 'ok')
('PPO', 'action_space', 'vector', 'obs_space', 'atari_ram', 'result', 'ok')
('PPO', 'action_space', 'vector', 'obs_space', 'discrete', 'result', 'ok')
('PPO', 'action_space', 'vector', 'obs_space', 'image', 'result', 'ok')
('PPO', 'action_space', 'vector', 'obs_space', 'mixed_tuple', 'result', 'ok')
('PPO', 'action_space', 'vector', 'obs_space', 'simple_tuple', 'result', 'ok')
('PPO', 'action_space', 'vector', 'obs_space', 'vector', 'result', 'ok')
Disconnecting client on fd 11
Disconnecting client on fd 74
Disconnecting client on fd 59
Disconnecting client on fd 58
Disconnecting client on fd 57
Disconnecting client on fd 56
[WARN] (/ray/src/global_scheduler/global_scheduler.cc:412) Missed too many heartbeats from local scheduler, marking as dead.
Disconnecting client on fd 55
Disconnecting client on fd 51
Disconnecting client on fd 54
Disconnecting client on fd 53
Disconnecting client on fd 52
Disconnecting client on fd 50
Disconnecting client on fd 49
Disconnecting client on fd 48
Disconnecting client on fd 47
Disconnecting client on fd 46
Disconnecting client on fd 45
Disconnecting client on fd 44
Disconnecting client on fd 43
Disconnecting client on fd 42
Disconnecting client on fd 41
Disconnecting client on fd 40
Disconnecting client on fd 39
Disconnecting client on fd 38
Disconnecting client on fd 37
Disconnecting client on fd 36
Disconnecting client on fd 35
Disconnecting client on fd 34
Disconnecting client on fd 33
Disconnecting client on fd 32
Disconnecting client on fd 31
Disconnecting client on fd 30
Disconnecting client on fd 29
Disconnecting client on fd 28
Disconnecting client on fd 27
Disconnecting client on fd 26
Disconnecting client on fd 25
Disconnecting client on fd 24
Disconnecting client on fd 23
Disconnecting client on fd 22
Disconnecting client on fd 21
Disconnecting client on fd 20
Disconnecting client on fd 19
Disconnecting client on fd 18
Disconnecting client on fd 17
Disconnecting client on fd 16
Disconnecting client on fd 15
Disconnecting client on fd 14
Disconnecting client on fd 13
Disconnecting client on fd 12
Disconnecting client on fd 9
Disconnecting client on fd 7
+ docker run --rm --shm-size=10G --memory=10G 9cba6202a7e5a1b8b52b05e0fdca82b57fb1f40abc4ec6a4b5dee6a149dcce35 python /ray/python/ray/tune/examples/tune_mnist_ray.py --fast
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
Allowing the Plasma store to use up to 8.58993GB of memory.
Starting object store with directory /dev/shm and huge page support disabled
2018-02-02 23:07:46.896542: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 63
2018-02-02 23:07:51.726365: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-02-02 23:07:51.819173: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Waiting for redis server at 127.0.0.1:40482 to respond...
Waiting for redis server at 127.0.0.1:15595 to respond...
Warning: Reducing object store memory because /dev/shm has only 10737418240 bytes available. You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you may need to pass an argument with the flag '--shm-size' to 'docker run'.
Starting local scheduler with the following resources: {'GPU': 0, 'CPU': 48}.

======================================================================
View the web UI at http://localhost:8888/notebooks/ray_ui87581.ipynb?token=92e4bfc2714eade11807910bc5bbd5fee93106644c70d1ae
======================================================================

== Status ==
Using FIFO scheduling algorithm.
Result logdir: /root/ray_results/tune_mnist_test
PENDING trials:
 - train_mnist_0_activation=relu:	PENDING
 - train_mnist_1_activation=elu:	PENDING
 - train_mnist_2_activation=tanh:	PENDING

Unified logger created with logdir '/root/ray_results/tune_mnist_test/train_mnist_0_activation=relu_2018-02-02_23-07-33GlOhOA'
WARNING: Serializing objects of type <class 'ray.tune.registry._Registry'> by expanding them as dictionaries of their fields. This behavior may be incorrect in some cases.
== Status ==
Using FIFO scheduling algorithm.
Resources used: 1/48 CPUs, 0/0 GPUs
Result logdir: /root/ray_results/tune_mnist_test
PENDING trials:
 - train_mnist_1_activation=elu:	PENDING
 - train_mnist_2_activation=tanh:	PENDING
RUNNING trials:
 - train_mnist_0_activation=relu:	RUNNING

Unified logger created with logdir '/root/ray_results/tune_mnist_test/train_mnist_1_activation=elu_2018-02-02_23-07-33_2txX1'
Unified logger created with logdir '/root/ray_results/tune_mnist_test/train_mnist_2_activation=tanh_2018-02-02_23-07-331PERKP'
TrainingResult for train_mnist_2_activation=tanh:
  date: 2018-02-02_23-07-47
  experiment_id: 4acc76e1e49a40c3a4fa374eccc7806e
  hostname: 470a9a978c93
  mean_accuracy: 0.14000000059604645
  pid: !!python/long '92'
  time_this_iter_s: 2.0022079944610596
  time_total_s: 2.0022079944610596
  timestamp: !!python/long '1517612867'
  timesteps_this_iter: !!python/long '0'
  timesteps_total: !!python/long '0'
  training_iteration: !!python/long '1'
  
== Status ==
Using FIFO scheduling algorithm.
Resources used: 3/48 CPUs, 0/0 GPUs
Result logdir: /root/ray_results/tune_mnist_test
RUNNING trials:
 - train_mnist_0_activation=relu:	RUNNING
 - train_mnist_1_activation=elu:	RUNNING
 - train_mnist_2_activation=tanh:	RUNNING [pid=92], 2 s, 0 ts, 0.14 acc

TrainingResult for train_mnist_2_activation=tanh:
  date: 2018-02-02_23-07-48
  done: true
  experiment_id: 4acc76e1e49a40c3a4fa374eccc7806e
  hostname: 470a9a978c93
  mean_accuracy: 0.7799999713897705
  pid: !!python/long '92'
  time_this_iter_s: 1.0011250972747803
  time_total_s: 3.00333309173584
  timestamp: !!python/long '1517612868'
  timesteps_this_iter: !!python/long '40'
  timesteps_total: !!python/long '40'
  training_iteration: !!python/long '2'
  
Stopping train_mnist_2_activation=tanh Actor timed out, but moving on...
TrainingResult for train_mnist_1_activation=elu:
  date: 2018-02-02_23-07-52
  experiment_id: be0ad41181f14d8caabcd4e907125b46
  hostname: 470a9a978c93
  mean_accuracy: 0.23999999463558197
  pid: !!python/long '90'
  time_this_iter_s: 7.007907867431641
  time_total_s: 7.007907867431641
  timestamp: !!python/long '1517612872'
  timesteps_this_iter: !!python/long '10'
  timesteps_total: !!python/long '10'
  training_iteration: !!python/long '1'
  
== Status ==
Using FIFO scheduling algorithm.
Resources used: 2/48 CPUs, 0/0 GPUs
Result logdir: /root/ray_results/tune_mnist_test
RUNNING trials:
 - train_mnist_0_activation=relu:	RUNNING
 - train_mnist_1_activation=elu:	RUNNING [pid=90], 7 s, 10 ts, 0.24 acc
TERMINATED trials:
 - train_mnist_2_activation=tanh:	TERMINATED [pid=92], 3 s, 40 ts, 0.78 acc

TrainingResult for train_mnist_0_activation=relu:
  date: 2018-02-02_23-07-52
  experiment_id: d6160b420efe42c385fdbe969c47f987
  hostname: 470a9a978c93
  mean_accuracy: 0.159999996423721Disconnecting client on fd 62
Disconnecting client on fd 61
Disconnecting client on fd 11
INFO:root:message-handler: driver_removed_handler
INFO:root:Driver 5a3c185ba5103e4ff8f9ceb64f45c61b70ae2ef1 has been removed.
Disconnecting client on fd 60
[WARN] (/ray/src/global_scheduler/global_scheduler.cc:412) Missed too many heartbeats from local scheduler, marking as dead.
Disconnecting client on fd 59
Disconnecting client on fd 58
Disconnecting client on fd 57
Disconnecting client on fd 56
Disconnecting client on fd 55
Disconnecting client on fd 54
Disconnecting client on fd 52
Disconnecting client on fd 53
Disconnecting client on fd 51
Disconnecting client on fd 49
Disconnecting client on fd 50
Disconnecting client on fd 48
Disconnecting client on fd 47
Disconnecting client on fd 46
Disconnecting client on fd 45
Disconnecting client on fd 44
Disconnecting client on fd 43
Disconnecting client on fd 42
Disconnecting client on fd 41
Disconnecting client on fd 40
Disconnecting client on fd 39
Disconnecting client on fd 38
Disconnecting client on fd 37
Disconnecting client on fd 36
Disconnecting client on fd 35
Disconnecting client on fd 34
Disconnecting client on fd 33
Disconnecting client on fd 32
Disconnecting client on fd 31
Disconnecting client on fd 29
Disconnecting client on fd 30
Disconnecting client on fd 28
Disconnecting client on fd 27
Disconnecting client on fd 26
Disconnecting client on fd 24
Disconnecting client on fd 25
Disconnecting client on fd 23
Disconnecting client on fd 22
Disconnecting client on fd 21
Disconnecting client on fd 20
Disconnecting client on fd 19
Disconnecting client on fd 18
Disconnecting client on fd 17
Disconnecting client on fd 16
Disconnecting client on fd 15
Disconnecting client on fd 14
Disconnecting client on fd 13
Disconnecting client on fd 9
Disconnecting client on fd 7
3
  pid: !!python/long '88'
  time_this_iter_s: 7.007331132888794
  time_total_s: 7.007331132888794
  timestamp: !!python/long '1517612872'
  timesteps_this_iter: !!python/long '0'
  timesteps_total: !!python/long '0'
  training_iteration: !!python/long '1'
  
TrainingResult for train_mnist_1_activation=elu:
  date: 2018-02-02_23-07-53
  done: true
  experiment_id: be0ad41181f14d8caabcd4e907125b46
  hostname: 470a9a978c93
  mean_accuracy: 0.3799999952316284
  pid: !!python/long '90'
  time_this_iter_s: 1.0010929107666016
  time_total_s: 8.009000778198242
  timestamp: !!python/long '1517612873'
  timesteps_this_iter: !!python/long '10'
  timesteps_total: !!python/long '20'
  training_iteration: !!python/long '2'
  
Stopping train_mnist_1_activation=elu Actor timed out, but moving on...
TrainingResult for train_mnist_0_activation=relu:
  date: 2018-02-02_23-07-53
  done: true
  experiment_id: d6160b420efe42c385fdbe969c47f987
  hostname: 470a9a978c93
  mean_accuracy: 0.47999998927116394
  pid: !!python/long '88'
  time_this_iter_s: 1.0011050701141357
  time_total_s: 8.00843620300293
  timestamp: !!python/long '1517612873'
  timesteps_this_iter: !!python/long '20'
  timesteps_total: !!python/long '20'
  training_iteration: !!python/long '2'
  
Stopping train_mnist_0_activation=relu Actor timed out, but moving on...
== Status ==
Using FIFO scheduling algorithm.
Resources used: 0/48 CPUs, 0/0 GPUs
Result logdir: /root/ray_results/tune_mnist_test
TERMINATED trials:
 - train_mnist_0_activation=relu:	TERMINATED [pid=88], 8 s, 20 ts, 0.48 acc
 - train_mnist_1_activation=elu:	TERMINATED [pid=90], 8 s, 20 ts, 0.38 acc
 - train_mnist_2_activation=tanh:	TERMINATED [pid=92], 3 s, 40 ts, 0.78 acc

+ docker run --rm --shm-size=10G --memory=10G 9cba6202a7e5a1b8b52b05e0fdca82b57fb1f40abc4ec6a4b5dee6a149dcce35 python /ray/python/ray/rllib/examples/multiagent_mountaincar.py
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
[2018-02-02 23:07:59,277] Making new env: MultiAgentMountainCarEnv-v0
2018-02-02 23:07:59.288236: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Waiting for redis server at 127.0.0.1:20497 to respond...
Waiting for redis server at 127.0.0.1:22997 to respond...
Warning: Reducing object store memory because /dev/shm has only 10737418240 bytes available. You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you may need to pass an argument with the flag '--shm-size' to 'docker run'.
Starting local scheduler with the following resources: {'GPU': 0, 'CPU': 4}.

======================================================================
View the web UI at http://localhost:8888/notebooks/ray_ui68569.ipynb?token=0fc25e1a1752103ddc5604de01349bb99d405283b9d31896
======================================================================

Unified logger created with logdir '/root/ray_results/2018-02-02_23-07-59Jy7jJL'
Observation shape is ((2,), (2,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(2,)
Observation shape is (2,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(2,)
Observation shape is (2,)
Not using any observation preprocessor.
Constructing fcnet [32, 32] <function tanh at 0x7f005847f668>
Constructing fcnet [32, 32] <function tanh at 0x7f005847f668>
Constructing fcnet [32, 32] <function tanh at 0x7f005847f668>
Constructing fcnet [32, 32] <function tanh at 0x7f005847f668>
Constructing fcnet [32, 32] <function tanh at 0x7f005847f668>
Constructing fcnet [32, 32] <function tanh at 0x7f005847f668>
Constructing fcnet [32, 32] <function tanh at 0x7f005847f668>
Constructing fcnet [32, 32] <function tanh at 0x7f005847f668>
Constructing fcnet [32, 32] <function tanh at 0x7f005847f668>
Constructing fcnet [32, 32] <function tanh at 0x7f005847f668>
WARNING: Serializing objects of type <class 'ray.tune.registry._Registry'> by expanding them as dictionaries of their fields. This behavior may be incorrect in some cases.
===> iteration 0
Computing policy (iterations=10, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0   -7.28629e-03   -7.28650e-03    0.00000e+00    1.09206e-06    2.19720e+00
              1   -7.59169e-03   -7.59305e-03    0.00000e+00    6.80414e-06    2.19720e+00
              2   -7.83105e-03   -7.83474e-03    0.00000e+00    1.84379e-05    2.19720e+00
              3   -8.09884e-03   -8.10601e-03    0.00000e+00    3.58431e-05    2.19719e+00
              4   -8.35336e-03   -8.36513e-03    0.00000e+00    5.88067e-05    2.19718e+00
              5   -8.57374e-03   -8.59155e-03    0.00000e+00    8.90922e-05    2.19716e+00
              6   -8.85832e-03   -8.88346e-03    0.00000e+00    1.25699e-04    2.19713e+00
              7   -9.09853e-03   -9.13197e-03    0.00000e+00    1.67187e-04    2.19710e+00
              8   -9.34568e-03   -9.38880e-03    0.00000e+00    2.15614e-04    2.19706e+00
              9   -9.60939e-03   -9.66363e-03    0.00000e+00    2.71160e-04    2.19701e+00
+ docker run --rm --shm-size=10G --memory=10G 9cba6202a7e5a1b8b52b05e0fdca82b57fb1f40abc4ec6a4b5dee6a149dcce35 python /ray/python/ray/rllib/examples/multiagent_pendulum.py
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
[2018-02-02 23:08:13,372] Making new env: MultiAgentPendulumEnv-v0
2018-02-02 23:08:13.383880: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Waiting for redis server at 127.0.0.1:27642 to respond...
Waiting for redis server at 127.0.0.1:42227 to respond...
Warning: Reducing object store memory because /dev/shm has only 10737418240 bytes available. You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you may need to pass an argument with the flag '--shm-size' to 'docker run'.
Starting local scheduler with the following resources: {'GPU': 0, 'CPU': 4}.

======================================================================
View the web UI at http://localhost:8888/notebooks/ray_ui73896.ipynb?token=4dd7fce7a39826cce60688d058905c72a352ac9320fe5dd2
======================================================================

Unified logger created with logdir '/root/ray_results/2018-02-02_23-08-13QH7Za9'
Observation shape is ((3,), (3,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(3,)
Observation shape is (3,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(3,)
Observation shape is (3,)
Not using any observation preprocessor.
Constructing fcnet [32, 32] <function tanh at 0x7f0a0f47f668>
Constructing fcnet [32, 32] <function tanh at 0x7f0a0f47f668>
Constructing fcnet [256, 256] <function tanh at 0x7f0a0f47f668>
Constructing fcnet [32, 32] <function tanh at 0x7f0a0f47f668>
Constructing fcnet [32, 32] <function tanh at 0x7f0a0f47f668>
Constructing fcnet [256, 256] <function tanh at 0x7f0a0f47f668>
Constructing fcnet [32, 32] <function tanh at 0x7f0a0f47f668>
Constructing fcnet [32, 32] <function tanh at 0x7f0a0f47f668>
Constructing fcnet [256, 256] <function tanh at 0x7f0a0f47f668>
Constructing fcnet [32, 32] <function tanh at 0x7f0a0f47f668>
Constructing fcnet [32, 32] <function tanh at 0x7f0a0f47f668>
Constructing fcnet [256, 256] <function tanh at 0x7f0a0f47f668>
Constructing fcnet [32, 32] <function tanh at 0x7f0a0f47f668>
Constructing fcnet [32, 32] <function tanh at 0x7f0a0f47f668>
Constructing fcnet [256, 256] <function tanh at 0x7f0a0f47f668>
WARNING: Serializing objects of type <class 'ray.tune.registry._Registry'> by expanding them as dictionaries of their fields. This behavior may be incorrect in some cases.
===> iteration 0
Computing policy (iterations=10, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.68580e+03   -1.15261e-02    1.68581e+03    3.08135e-06    2.83811e+00
              1    1.68503e+03   -1.20082e-02    1.68504e+03    1.29302e-05    2.83870e+00
              2    1.68423e+03   -1.25341e-02    1.68425e+03    3.28445e-05    2.83925e+00
              3    1.68335e+03   -1.29520e-02    1.68336e+03    6.44869e-05    2.83981e+00
              4    1.68232e+03   -1.34137e-02    1.68233e+03    1.07926e-04    2.84043e+00
              5    1.68104e+03   -1.38345e-02    1.68106e+03    1.63046e-04    2.84100e+00
              6    1.67972e+03   -1.43189e-02    1.67973e+03    2.32542e-04    2.84157e+00
              7    1.67785e+03   -1.47633e-02    1.67786e+03    3.11507e-04    2.84218e+00
              8    1.67569e+03   -1.52137e-02    1.67571e+03    3.97957e-04    2.84273e+00
              9    1.67288e+03   -1.55613e-02    1.67290e+03    4.92832e-04    2.84331e+00
Notifying endpoint 'HTTP:'
ERROR: Failed to notify endpoint 'HTTP:'
java.net.MalformedURLException: no protocol: 
	at java.net.URL.<init>(URL.java:593)
	at java.net.URL.<init>(URL.java:490)
	at java.net.URL.<init>(URL.java:439)
	at com.tikal.hudson.plugins.notification.Protocol$3.send(Protocol.java:61)
	at com.tikal.hudson.plugins.notification.Phase.handle(Phase.java:45)
	at com.tikal.hudson.plugins.notification.JobListener.onCompleted(JobListener.java:36)
	at hudson.model.listeners.RunListener.fireCompleted(RunListener.java:201)
	at hudson.model.Run.execute(Run.java:1783)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:410)
Failed to notify endpoint 'HTTP:' - java.net.MalformedURLException: no protocol: 
Finished: SUCCESS