SuccessConsole Output

Skipping 242 KB.. Full Log
e-01
              4    3.34422e+02   -2.52508e-02    3.34446e+02    8.58718e-03    6.84286e-01
current status: TrainingResult(timesteps_total=6102, done=None, info={'sgd_time': 0.45528531074523926, 'sample_throughput': 2295.2640362797542, 'shuffle_time': 0.00033092498779296875, 'kl_divergence': 0.0085871816, 'rollouts_time': 3.183277130126953, 'load_time': 0.04343008995056152, 'kl_coefficient': 0.2}, episode_reward_mean=22.516605166051662, episode_len_mean=22.516605166051662, episodes_total=None, mean_accuracy=None, mean_validation_accuracy=None, mean_loss=None, neg_mean_loss=None, experiment_id='00f1a2d7ce9548a6bbeccd4afc378137', training_iteration=1, timesteps_this_iter=6102, time_this_iter_s=3.691253900527954, time_total_s=3.691253900527954, pid=1, date='2018-02-02_23-48-51', timestamp=1517615331, hostname='ac3668b67976')
===> iteration 1
Computing policy (iterations=5, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    5.82881e+02   -4.85083e-03    5.82886e+02    2.04268e-04    6.80460e-01
              1    5.82459e+02   -1.24750e-02    5.82472e+02    1.67419e-03    6.73577e-01
              2    5.81983e+02   -1.98813e-02    5.82002e+02    5.15878e-03    6.64412e-01
              3    5.81557e+02   -2.82026e-02    5.81583e+02    1.07903e-02    6.53271e-01
              4    5.80983e+02   -3.35830e-02    5.81013e+02    1.70937e-02    6.42834e-01
current status: TrainingResult(timesteps_total=12299, done=None, info={'sgd_time': 0.15651774406433105, 'sample_throughput': 6682.948353575035, 'shuffle_time': 0.0003349781036376953, 'kl_divergence': 0.017093688, 'rollouts_time': 1.459928035736084, 'load_time': 0.001867055892944336, 'kl_coefficient': 0.2}, episode_reward_mean=30.678217821782177, episode_len_mean=30.678217821782177, episodes_total=None, mean_accuracy=None, mean_validation_accuracy=None, mean_loss=None, neg_mean_loss=None, experiment_id='00f1a2d7ce9548a6bbeccd4afc378137', training_iteration=2, timesteps_this_iter=6197, time_this_iter_s=1.6268420219421387, time_total_s=5.318095922470093, pid=1, date='2018-02-02_23-48-53', timestamp=1517615333, hostname='ac3668b67976')
===> iteration 2
Computing policy (iterations=5, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    2.27956e+03   -1.97577e-03    2.27956e+03    5.12260e-05    6.52671e-01
              1    2.27885e+03   -4.50704e-03    2.27885e+03    5.03229e-04    6.46506e-01
              2    2.27809e+03   -7.79949e-03    2.27810e+03    1.74004e-03    6.38082e-01
              3    2.27674e+03   -1.09482e-02    2.27675e+03    3.88016e-03    6.28619e-01
              4    2.27388e+03   -1.37455e-02    2.27389e+03    6.82151e-03    6.18857e-01
current status: TrainingResult(timesteps_total=18530, done=None, info={'sgd_time': 0.18241000175476074, 'sample_throughput': 5723.370374194696, 'shuffle_time': 0.0003600120544433594, 'kl_divergence': 0.0068215081, 'rollouts_time': 1.5719268321990967, 'load_time': 0.002153158187866211, 'kl_coefficient': 0.2}, episode_reward_mean=58.783018867924525, episode_len_mean=58.783018867924525, episodes_total=None, mean_accuracy=None, mean_validation_accuracy=None, mean_loss=None, neg_mean_loss=None, experiment_id='00f1a2d7ce9548a6bbeccd4afc378137', training_iteration=3, timesteps_this_iter=6231, time_this_iter_s=1.7675859928131104, time_total_s=7.085681915283203, pid=1, date='2018-02-02_23-48-55', timestamp=1517615335, hostname='ac3668b67976')
Saving checkpoint to object store, 6968065 bytes
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7f7763e88f50> [ 0.74048585  0.12821085  0.02655059  0.02241079] 0.453 0.4595
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7f7763e88f50> [ 0.40064568  0.26076949  0.27314942  0.62518549] 0.76 0.732
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7f7763e88f50> [[2018-02-02 23:49:28,006] Making new env: CartPole-v0
Disconnecting client on fd 63
Disconnecting client on fd 68
Disconnecting client on fd 67
Disconnecting client on fd 65
Disconnecting client on fd 69
Disconnecting client on fd 80
Disconnecting client on fd 64
Disconnecting client on fd 81
Disconnecting client on fd 79
Disconnecting client on fd 62
[2018-02-02 23:49:28,881] Making new env: CartPole-v0
[2018-02-02 23:49:34,369] Making new env: CartPole-v0
[2018-02-02 23:49:34,497] Making new env: CartPole-v0
[2018-02-02 23:49:34,605] Making new env: CartPole-v0
[2018-02-02 23:49:34,643] Making new env: CartPole-v0
2018-02-02 23:49:34.765777: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
[2018-02-02 23:49:34,815] Making new env: CartPole-v0
2018-02-02 23:49:34.843005: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
[2018-02-02 23:49:34,942] Making new env: CartPole-v0
2018-02-02 23:49:34.962379: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
[2018-02-02 23:49:35,011] Making new env: CartPole-v0
2018-02-02 23:49:35.019770: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
[2018-02-02 23:49:35,063] Making new env: CartPole-v0
2018-02-02 23:49:35.196935: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-02-02 23:49:35.285569: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-02-02 23:49:35.352835: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-02-02 23:49:35.410112: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
 0.24844181  0.87126847  0.32584486  0.96475781] 0.752 0.7555
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7f7763e88f50> [ 0.99476425  0.13068767  0.7099356   0.04996793] 0.7475 0.723
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7f7763e88f50> [ 0.54831344  0.26791225  0.33752149  0.49000328] 0.7485 0.739
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7f7763e88f50> [ 0.95131402  0.68872639  0.19095092  0.12934479] 0.594 0.586
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7f7763e88f50> [ 0.40367383  0.23926888  0.85838904  0.31258727] 0.8015 0.7715
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7f7763e88f50> [ 0.78959276  0.9364339   0.66228908  0.41188354] 0.738 0.7475
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7f7763e88f50> [ 0.93278486  0.01293606  0.36116991  0.9502263 ] 0.6965 0.718
Checking computed actions <ray.rllib.ppo.ppo.PPOAgent object at 0x7f7763e88f50> [ 0.75923186  0.77447556  0.27009862  0.49862162] 0.658 0.667
Unified logger created with logdir '/root/ray_results/2018-02-02_23-49-28wEW7jv'
Observation shape is (4,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f7949329f50>
Setting up loss
Unified logger created with logdir '/root/ray_results/2018-02-02_23-49-280hSDhC'
Observation shape is (4,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7f7949329f50>
Setting up loss
current status: TrainingResult(timesteps_total=2255, done=None, info={}, episode_reward_mean=24.510869565217391, episode_len_mean=24.510869565217391, episodes_total=None, mean_accuracy=None, mean_validation_accuracy=None, mean_loss=None, neg_mean_loss=None, experiment_id='f67eb14d0af44def9dc062d174c4c51b', training_iteration=1, timesteps_this_iter=2255, time_this_iter_s=6.306046962738037, time_total_s=6.306046962738037, pid=1, date='2018-02-02_23-49-35', timestamp=1517615375, hostname='ac3668b67976')
current status: TrainingResult(timesteps_total=4748, done=None, info={}, episode_reward_mean=28.011235955056179, episode_len_mean=28.011235955056179, episodes_total=None, mean_accuracy=None, mean_validation_accuracy=None, mean_loss=None, neg_mean_loss=None, experiment_id='f67eb14d0af44def9dc062d174c4c51b', training_iteration=2, timesteps_this_iter=2493, time_this_iter_s=0.5112431049346924, time_total_s=6.8172900676727295, pid=1, date='2018-02-02_23-49-36', timestamp=1517615376, hostname='ac3668b67976')
current status: TrainingResult(timesteps_total=7340, done=None, info={}, episode_reward_mean=37.028571428571432, episode_len_mean=37.028571428571432, episodes_total=None, mean_accuracy=None, mean_validation_accuracy=None, mean_loss=None, neg_mean_loss=None, experiment_id='f67eb14d0af44def9dc062d174c4c51b', training_iteration=3, timesteps_this_iter=2592, time_this_iter_s=0.4921529293060303, time_total_s=7.30944299697876, pid=1, date='2018-02-02_23-49-36', timestamp=1517615376, hostname='ac3668b67976')
Saving checkpoint to object store, 6356112 bytes
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7f77a013b110> [ 0.56634805  0.53886955  0.53440958  0.94817742] 0.6645 0.632
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7f77a013b110> [ 0.74970177  0.94567675  0.70207506  0.50852632] 0.54 0.541
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7f77a013b110> [ 0.40903584  0.71565908  0.22967949  0.42569819] 0.526 0.5255
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7f77a013b110> [ 0.98425926  0.74160812  0.30300454  0.5613418 ] 0.5485 0.568
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7f77a013b110> [ 0.90806478  0.83700524  0.48993016  0.03109895] 0.4465 0.457
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7f77a013b110> [ 0.10011939  0.84653185  0.07251385  0.1015167 ] 0.464 0.4785
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7f77a013b110> [ 0.12717215  0.21752273  0.5937027   0.92734739] 0.6735 0.664
Checking comDisconnecting client on fd 11
Disconnecting client on fd 62
Disconnecting client on fd 64
Disconnecting client on fd 63
Disconnecting client on fd 79
Disconnecting client on fd 65
Disconnecting client on fd 69
Disconnecting client on fd 59
Disconnecting client on fd 67
Disconnecting client on fd 68
Disconnecting client on fd 58
Disconnecting client on fd 57
Disconnecting client on fd 56
[WARN] (/ray/src/global_scheduler/global_scheduler.cc:412) Missed too many heartbeats from local scheduler, marking as dead.
Disconnecting client on fd 55
Disconnecting client on fd 54
Disconnecting client on fd 52
Disconnecting client on fd 51
Disconnecting client on fd 53
Disconnecting client on fd 50
Disconnecting client on fd 49
Disconnecting client on fd 47
Disconnecting client on fd 48
Disconnecting client on fd 46
Disconnecting client on fd 44
Disconnecting client on fd 45
Disconnecting client on fd 43
Disconnecting client on fd 42
Disconnecting client on fd 41
Disconnecting client on fd 40
Disconnecting client on fd 39
Disconnecting client on fd 38
Disconnecting client on fd 37
Disconnecting client on fd 36
Disconnecting client on fd 35
Disconnecting client on fd 34
Disconnecting client on fd 33
Disconnecting client on fd 32
Disconnecting client on fd 31
Disconnecting client on fd 30
Disconnecting client on fd 29
Disconnecting client on fd 28
Disconnecting client on fd 27
Disconnecting client on fd 26
Disconnecting client on fd 25
Disconnecting client on fd 24
Disconnecting client on fd 23
Disconnecting client on fd 22
Disconnecting client on fd 21
Disconnecting client on fd 20
Disconnecting client on fd 19
Disconnecting client on fd 18
Disconnecting client on fd 17
Disconnecting client on fd 16
Disconnecting client on fd 15
Disconnecting client on fd 14
Disconnecting client on fd 13
Disconnecting client on fd 12
Disconnecting client on fd 9
Disconnecting client on fd 7
puted actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7f77a013b110> [ 0.15756552  0.61147796  0.90829407  0.0851171 ] 0.511 0.511
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7f77a013b110> [ 0.4020069   0.8301583   0.35844502  0.2058719 ] 0.4905 0.4815
Checking computed actions <ray.rllib.a3c.a3c.A3CAgent object at 0x7f77a013b110> [ 0.64214035  0.13096786  0.31352419  0.95064576] 0.6715 0.6675
All checkpoint restore tests passed!
+ docker run --rm --shm-size=10G --memory=10G f92f28c594d96220fa3e04f4e1c18a53a5eec4680ab2f9ab421ae5f4543a7978 python /ray/python/ray/rllib/test/test_supported_spaces.py
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
testAll (__main__.ModelSupportedSpaces) ... Allowing the Plasma store to use up to 8.58993GB of memory.
Starting object store with directory /dev/shm and huge page support disabled
2018-02-02 23:50:15.736309: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:50:15,822] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:50:15,933] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:50:16,800] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:50:16,912] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:50:17,681] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:50:17,960] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:50:18,841] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:50:18,953] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:50:19,804] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:50:19,914] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:50:20,756] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:50:20,866] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:50:21,612] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
[2018-02-02 23:50:21,751] VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
Waiting for redis server at 127.0.0.1:44305 to respond...
Waiting for redis server at 127.0.0.1:37336 to respond...
Warning: Reducing object store memory because /dev/shm has only 10737418240 bytes available. You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you may need to pass an argument with the flag '--shm-size' to 'docker run'.
Starting local scheduler with the following resources: {'GPU': 0, 'CPU': 48}.

======================================================================
View the web UI at http://localhost:8888/notebooks/ray_ui6261.ipynb?token=bbdf1b1a6b018240adf0dc558f4f2590d432f1e5c8e1f3a3
======================================================================

('=== Testing', 'DQN', Discrete(5), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-15uygCp8'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
ok
()
('=== Testing', 'DQN', Discrete(5), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-16kvyi9t'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
ok
()
('=== Testing', 'DQN', Discrete(5), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-17G9Z9RL'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
ok
()
('=== Testing', 'DQN', Discrete(5), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-18e7CA0J'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
ok
()
('=== Testing', 'DQN', Discrete(5), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-19xok347'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
ok
()
('=== Testing', 'DQN', Discrete(5), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-20nqOHBZ'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
ok
()
('=== Testing', 'DQN', Discrete(5), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-21d06gAE'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
ok
()
('=== Testing', 'DQN', Box(5,), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22ijANT7'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', Box(5,), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22pPA1ts'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
unsupported
()
('=== Testing', 'DQN', Box(5,), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-223xPWjL'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', Box(5,), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22fCckG9'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
unsupported
()
('=== Testing', 'DQN', Box(5,), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22c7wTmf'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', Box(5,), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22k1Zr9c'
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', Box(5,), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22TobPFW'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
unsupported
()
('=== Testing', 'DQN', [Box(5,), Box(5,)], Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-224MjTn1'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', [Box(5,), Box(5,)], Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22cz5fLq'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
unsupported
()
('=== Testing', 'DQN', [Box(5,), Box(5,)], Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22P6lBku'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', [Box(5,), Box(5,)], Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22istpLD'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
unsupported
()
('=== Testing', 'DQN', [Box(5,), Box(5,)], Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22_iDHhI'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', [Box(5,), Box(5,)], Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22I4R_uy'
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', [Box(5,), Box(5,)], Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22ssi7t1'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
unsupported
()
('=== Testing', 'DQN', Tuple(Box(5,), Box(5,)), Tuple(Discrete(10), Box(5,)), '==='2018-02-02 23:50:41.096570: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 62
2018-02-02 23:50:53.437157: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 64
2018-02-02 23:50:57.218880: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 65
2018-02-02 23:51:00.998217: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 65
2018-02-02 23:51:04.744465: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
)
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22QGhKYa'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', Tuple(Box(5,), Box(5,)), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22DaLLJv'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
unsupported
()
('=== Testing', 'DQN', Tuple(Box(5,), Box(5,)), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-225iWRU0'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', Tuple(Box(5,), Box(5,)), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22GfoCKq'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
unsupported
()
('=== Testing', 'DQN', Tuple(Box(5,), Box(5,)), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22pMmy0m'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', Tuple(Box(5,), Box(5,)), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22qEYWQj'
Observation shape is (5,)
Not using any observation preprocessor.
unsupported
()
('=== Testing', 'DQN', Tuple(Box(5,), Box(5,)), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22fITHNG'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
unsupported
()
('=== Testing', 'A3C', Discrete(5), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-22qPZuei'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
WARNING: Serializing objects of type <class 'ray.tune.registry._Registry'> by expanding them as dictionaries of their fields. This behavior may be incorrect in some cases.
ok
()
('=== Testing', 'A3C', Discrete(5), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-41EvwnxB'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
ok
()
('=== Testing', 'A3C', Discrete(5), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-53QTteul'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
Setting up loss
ok
()
('=== Testing', 'A3C', Discrete(5), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-50-57P0UEHN'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
ok
()
('=== Testing', 'A3C', Discrete(5), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-01zU7sXU'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
ok
()
('Disconnecting client on fd 65
2018-02-02 23:51:08.870788: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 65
2018-02-02 23:51:12.802476: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:51:16.632343: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:51:20.548606: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:51:24.533144: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:51:28.465885: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:51:32.399418: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:51:36.790716: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:51:40.919663: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
=== Testing', 'A3C', Discrete(5), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-043cjVKb'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
ok
()
('=== Testing', 'A3C', Discrete(5), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-08id1fwJ'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
Setting up loss
ok
()
('=== Testing', 'A3C', Box(5,), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-13MX5GZE'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
ok
()
('=== Testing', 'A3C', Box(5,), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-16fOiPfu'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
ok
()
('=== Testing', 'A3C', Box(5,), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-20963EVk'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
Setting up loss
ok
()
('=== Testing', 'A3C', Box(5,), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-24RYlbLO'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
ok
()
('=== Testing', 'A3C', Box(5,), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-28MXwARO'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
ok
()
('=== Testing', 'A3C', Box(5,), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-32ncUVW9'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
ok
()
('=== Testing', 'A3C', Box(5,), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-36U61eyP'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
Setting up loss
ok
()
('=== Testing', 'A3C', [Box(5,), Box(5,)], Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-41XC9o8E'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', [Box(5,), Box(5,)], Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-41lfddX_'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', [Box(5,), Box(5,)], Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-41wQWwpt'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', [Box(5,), Box(5,)], Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-41dXmp4C'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', [Box(5,), Box(5,)], Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-41QCsrWB'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', [Box(5,), Box(5,)], Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-41YzyWjv'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', [Box(5,), Box(5,)], Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-41_yEfq8'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', Tuple(Box(5,), Box(5,)), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-41XIucQ9'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', Tuple(Box(5,), Box(5,)), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-42Lpdvsz'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', Tuple(Box(5,), Box(5,)), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-42qpKCL3'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', Tuple(Box(5,), Box(5,)), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-42sEsMJX'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', Tuple(Box(5,), Box(5,)), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-42NZzG9o'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', Tuple(Box(5,), Box(5,)), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-42_x0mwe'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'A3C', Tuple(Box(5,), Box(5,)), Box(210, 160, 3), '===')
Unified logger created with logdir 2018-02-02 23:51:46.490608: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:51:51.574825: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 67
2018-02-02 23:51:56.781906: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
'/root/ray_results/2018-02-02_23-51-428k0eIH'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
Setting up loss
'NotImplementedType' object is not callable
Traceback (most recent call last):
  File "/ray/python/ray/rllib/test/test_supported_spaces.py", line 84, in check_support
    a = get_agent_class(alg)(config=config, env="stub_env")
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 82, in __init__
    Trainable.__init__(self, config, registry, logger_creator)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/tune/trainable.py", line 88, in __init__
    self._setup()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/agent.py", line 105, in _setup
    self._init()
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c.py", line 74, in _init
    start_sampler=False)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/a3c_evaluator.py", line 37, in __init__
    registry, env.observation_space.shape, env.action_space, config)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/shared_model.py", line 18, in __init__
    registry, ob_space, ac_space, config, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 27, in __init__
    self.setup_loss(action_space)
  File "/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/a3c/tfpolicy.py", line 43, in setup_loss
    "currently not supported")
TypeError: 'NotImplementedType' object is not callable

ERROR
()
('=== Testing', 'PPO', Discrete(5), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-423qhCoA'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.00312e+00    0.00000e+00    1.00312e+00    0.00000e+00    1.60943e+00
ok
()
('=== Testing', 'PPO', Discrete(5), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-47n6rHRl'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.98608e-01    0.00000e+00    9.98608e-01    0.00000e+00    1.60943e+00
ok
()
('=== Testing', 'PPO', Discrete(5), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-52Pk0fB1'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.05329e+00    0.00000e+00    1.05329e+00    0.00000e+00    1.60772e+00
ok
()
('=== Testing', 'PPO', Discrete(5), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-51-58ArBM_K'
Observation shape is (128,)
Assuming Atari ram envDisconnecting client on fd 70
2018-02-02 23:52:02.163757: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:52:07.012700: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:52:12.126598: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:52:17.219172: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:52:22.698375: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.94742e-01    0.00000e+00    9.94742e-01    0.00000e+00    1.60943e+00
ok
()
('=== Testing', 'PPO', Discrete(5), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-52-03OAW4oa'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.90725e-01    0.00000e+00    9.90725e-01    0.00000e+00    1.60943e+00
ok
()
('=== Testing', 'PPO', Discrete(5), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-52-08EqvFke'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.00126e+00    0.00000e+00    1.00126e+00    0.00000e+00    1.60943e+00
ok
()
('=== Testing', 'PPO', Discrete(5), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-52-13QtLDGa'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.12526e+00    0.00000e+00    1.12526e+00   -2.43944e-08    1.60910e+00
ok
()
('=== Testing', 'PPO', Box(5,), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-52-18YgUxlB'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.00680e+00    0.00000e+00    1.00680e+00    0.00000e+00    7.09636e+00
ok
()
('=== Testing', 'PPO', Box(5,), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-52-23YnMVia'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iterat2018-02-02 23:52:27.512981: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:52:32.427305: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:52:37.440310: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:52:42.739853: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:52:47.505659: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:52:52.817642: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
ion 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.01179e+00    0.00000e+00    1.01179e+00    0.00000e+00    7.09529e+00
ok
()
('=== Testing', 'PPO', Box(5,), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-52-28OgO_fN'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.45741e-01    0.00000e+00    9.45741e-01    0.00000e+00    7.08600e+00
ok
()
('=== Testing', 'PPO', Box(5,), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-52-33dfG88b'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.91882e-01    0.00000e+00    9.91882e-01    0.00000e+00    7.09513e+00
ok
()
('=== Testing', 'PPO', Box(5,), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-52-38nd4xAf'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.01153e+00    0.00000e+00    1.01153e+00    0.00000e+00    7.11237e+00
ok
()
('=== Testing', 'PPO', Box(5,), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-52-43O05coS'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.01446e+00    0.00000e+00    1.01446e+00    0.00000e+00    7.11645e+00
ok
()
('=== Testing', 'PPO', Box(5,), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-52-48OMA1vB'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.50122e-01    0.00000e+00    9.50122e-01    0.00000e+00    6.79151e+00
ok
()
('=== Testing', 'PPO', [Box(5,), Box(5,)], Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-52-54q58Duv'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <f2018-02-02 23:52:58.541840: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:53:03.848492: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 70
2018-02-02 23:53:09.552385: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:53:15.455486: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:53:21.096805: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:53:26.678796: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
unction tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.97961e-01    0.00000e+00    9.97961e-01    0.00000e+00    1.41870e+01
ok
()
('=== Testing', 'PPO', [Box(5,), Box(5,)], Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-52-598Fk_xh'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.00600e+00    0.00000e+00    1.00600e+00    0.00000e+00    1.42260e+01
ok
()
('=== Testing', 'PPO', [Box(5,), Box(5,)], Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-53-05W_KMCp'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.75897e-01    0.00000e+00    9.75897e-01    0.00000e+00    1.41309e+01
ok
()
('=== Testing', 'PPO', [Box(5,), Box(5,)], Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-53-11qDmpON'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.93667e-01    0.00000e+00    9.93667e-01    0.00000e+00    1.42274e+01
ok
()
('=== Testing', 'PPO', [Box(5,), Box(5,)], Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-53-16h4J6yq'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.01850e+00    0.00000e+00    1.01850e+00    0.00000e+00    1.41733e+01
ok
()
('=== Testing', 'PPO', [Box(5,), Box(5,)], Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-53-227Jdlxh'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.87265e-01    0.00000e+00    9.87265e-01    0.00000e+00    1.42121e+01
ok
()
('=== Testing', 'PPO', [Box(5,), Box(5,)], Box(210, 160, 3), '===')
Unified logger createdDisconnecting client on fd 71
2018-02-02 23:53:32.316759: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:53:37.989459: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:53:43.311725: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:53:49.165127: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:53:54.554478: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:53:59.842424: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
 with logdir '/root/ray_results/2018-02-02_23-53-28k3AOuZ'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.06653e+00    0.00000e+00    1.06653e+00    0.00000e+00    1.41253e+01
ok
()
('=== Testing', 'PPO', Tuple(Box(5,), Box(5,)), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-53-33WN_YrX'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.00752e+00    0.00000e+00    1.00752e+00    0.00000e+00    1.42025e+01
ok
()
('=== Testing', 'PPO', Tuple(Box(5,), Box(5,)), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-53-39y1h8iM'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.93273e-01    0.00000e+00    9.93273e-01    0.00000e+00    1.42041e+01
ok
()
('=== Testing', 'PPO', Tuple(Box(5,), Box(5,)), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-53-44eEs9lm'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.52306e-01    0.00000e+00    9.52306e-01    0.00000e+00    1.42363e+01
ok
()
('=== Testing', 'PPO', Tuple(Box(5,), Box(5,)), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-53-50TOV3YN'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.87615e-01    0.00000e+00    9.87615e-01    0.00000e+00    1.41663e+01
ok
()
('=== Testing', 'PPO', Tuple(Box(5,), Box(5,)), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-53-55VYZV1n'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter  Disconnecting client on fd 71
2018-02-02 23:54:04.723019: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:54:10.112648: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 71
2018-02-02 23:54:15.344397: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/es/es.py:262: RuntimeWarning: Mean of empty slice.
  tlogger.record_tabular("EvalEpRewMean", eval_returns.mean())
/opt/conda/lib/python2.7/site-packages/numpy/core/_methods.py:80: RuntimeWarning: invalid value encountered in double_scalars
  ret = ret.dtype.type(ret / rcount)
/opt/conda/lib/python2.7/site-packages/numpy/core/_methods.py:135: RuntimeWarning: Degrees of freedom <= 0 for slice
  keepdims=keepdims)
/opt/conda/lib/python2.7/site-packages/numpy/core/_methods.py:105: RuntimeWarning: invalid value encountered in true_divide
  arrmean, rcount, out=arrmean, casting='unsafe', subok=False)
/opt/conda/lib/python2.7/site-packages/numpy/core/_methods.py:127: RuntimeWarning: invalid value encountered in double_scalars
  ret = ret.dtype.type(ret / rcount)
/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/es/es.py:264: RuntimeWarning: Mean of empty slice.
  tlogger.record_tabular("EvalEpLenMean", eval_lengths.mean())
   total loss    policy loss        vf loss             kl        entropy
              0    1.00319e+00    0.00000e+00    1.00319e+00    0.00000e+00    1.42066e+01
ok
()
('=== Testing', 'PPO', Tuple(Box(5,), Box(5,)), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-54-01PLeHHu'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.95818e-01    0.00000e+00    9.95818e-01    0.00000e+00    1.42017e+01
ok
()
('=== Testing', 'PPO', Tuple(Box(5,), Box(5,)), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-54-051u1xvk'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
===> iteration 0
Computing policy (iterations=1, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    9.84527e-01    0.00000e+00    9.84527e-01    0.00000e+00    1.42973e+01
ok
()
('=== Testing', 'ES', Discrete(5), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-54-11wTFDmW'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 90.9     |
| UpdateRatio         | 0.118    |
| EpisodesThisIter    | 8        |
| EpisodesSoFar       | 8        |
| TimestepsThisIter   | 8        |
| TimestepsSoFar      | 8        |
| TimeElapsedThisIter | 3.16     |
| TimeElapsed         | 3.16     |
----------------------------------
/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/es/es.py:296: RuntimeWarning: Mean of empty slice.
  episode_reward_mean=eval_returns.mean(),
/opt/conda/lib/python2.7/site-packages/ray-0.3.0-py2.7-linux-x86_64.egg/ray/rllib/es/es.py:297: RuntimeWarning: Mean of empty slice.
  episode_len_mean=eval_lengths.mean(),
Disconnecting client on fd 71
2018-02-02 23:54:19.204013: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Discrete(5), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-54-157009Cu'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 517      |
| GradNorm            | 42.1     |
| UpdateRatio         | 0.116    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 3.05     |
| TimeElapsed         | 3.05     |
----------------------------------
Disconnecting client on fd 72
2018-02-02 23:54:23.013308: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Discrete(5), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-54-19p3rIa0'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 258      |
| GradNorm            | 5.5e+03  |
| UpdateRatio         | 1.33     |
| EpisodesThisIter    | 6        |
| EpisodesSoFar       | 6        |
| TimestepsThisIter   | 6        |
| TimestepsSoFar      | 6        |
| TimeElapsedThisIter | 3.19     |
| TimeElapsed         | 3.19     |
----------------------------------
Disconnecting client on fd 72
2018-02-02 23:54:27.219295: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Discrete(5), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-54-23qtzZSM'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 520      |
| GradNorm            | 61.7     |
| UpdateRatio         | 0.14     |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 3.33     |
| TimeElapsed         | 3.33     |
----------------------------------
Disconnecting client on fd 72
2018-02-02 23:54:31.099519: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Discrete(5), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-54-27zXuCIo'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 516      |
| GradNorm            | 23.8     |
| UpdateRatio         | 0.117    |
| EpisodesThisIter    | 12       |
| EpisodesSoFar       | 12       |
| TimestepsThisIter   | 12       |
| TimestepsSoFar      | 12       |
| TimeElapsedThisIter | 3.16     |
| TimeElapsed         | 3.16     |
----------------------------------
Disconnecting client on fd 72
2018-02-02 23:54:35.024120: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Discrete(5), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-54-31G6dT_U'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 87.5     |
| UpdateRatio         | 0.116    |
| EpisodesThisIter    | 8        |
| EpisodesSoFar       | 8        |
| TimestepsThisIter   | 8        |
| TimestepsSoFar      | 8        |
| TimeElapsedThisIter | 3.2      |
| TimeElapsed         | 3.2      |
----------------------------------
Disconnecting client on fd 72
2018-02-02 23:54:38.944376: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Discrete(5), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-54-35z1tznH'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 262      |
| GradNorm            | 5.51e+03 |
| UpdateRatio         | 1.31     |
| EpisodesThisIter    | 6        |
| EpisodesSoFar       | 6        |
| TimestepsThisIter   | 6        |
| TimestepsSoFar      | 6        |
| TimeElapsedThisIter | 3.41     |
| TimeElapsed         | 3.41     |
----------------------------------
Disconnecting client on fd 72
2018-02-02 23:54:43.001114: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Box(5,), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-54-39X_Y5ng'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 517      |
| GradNorm            | 43.7     |
| UpdateRatio         | 0.118    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 3.22     |
| TimeElapsed         | 3.22     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:54:46.867130: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Box(5,), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-54-43etlBDs'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 515      |
| GradNorm            | 9.49     |
| UpdateRatio         | 0.116    |
| EpisodesThisIter    | 16       |
| EpisodesSoFar       | 16       |
| TimestepsThisIter   | 16       |
| TimestepsSoFar      | 16       |
| TimeElapsedThisIter | 3.16     |
| TimeElapsed         | 3.16     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:54:50.594586: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Box(5,), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-54-4748wha8'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 258      |
| GradNorm            | 5.51e+03 |
| UpdateRatio         | 1.33     |
| EpisodesThisIter    | 6        |
| EpisodesSoFar       | 6        |
| TimestepsThisIter   | 6        |
| TimestepsSoFar      | 6        |
| TimeElapsedThisIter | 3.16     |
| TimeElapsed         | 3.16     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:54:54.562771: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Box(5,), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-54-51DeSBis'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 520      |
| GradNorm            | 34.8     |
| UpdateRatio         | 0.14     |
| EpisodesThisIter    | 12       |
| EpisodesSoFar       | 12       |
| TimestepsThisIter   | 12       |
| TimestepsSoFar      | 12       |
| TimeElapsedThisIter | 3.1      |
| TimeElapsed         | 3.1      |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:54:58.427363: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Box(5,), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-54-54riYcUx'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 42.9     |
| UpdateRatio         | 0.117    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 3.12     |
| TimeElapsed         | 3.12     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:55:02.313885: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Box(5,), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-54-584ZAKWI'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 517      |
| GradNorm            | 42.8     |
| UpdateRatio         | 0.116    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 3.3      |
| TimeElapsed         | 3.3      |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:55:06.176433: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Box(5,), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-55-02iTr_8v'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 262      |
| GradNorm            | 2.3e+04  |
| UpdateRatio         | 1.3      |
| EpisodesThisIter    | 4        |
| EpisodesSoFar       | 4        |
| TimestepsThisIter   | 4        |
| TimestepsSoFar      | 4        |
| TimeElapsedThisIter | 3.35     |
| TimeElapsed         | 3.35     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:55:10.244861: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', [Box(5,), Box(5,)], Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-55-06mZyeER'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 46.4     |
| UpdateRatio         | 0.121    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 2.95     |
| TimeElapsed         | 2.95     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:55:14.050093: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', [Box(5,), Box(5,)], Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-55-10yqCsxd'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 44.9     |
| UpdateRatio         | 0.119    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 3.11     |
| TimeElapsed         | 3.11     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:55:17.995477: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', [Box(5,), Box(5,)], Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-55-14vfxelp'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 287      |
| GradNorm            | 5.52e+03 |
| UpdateRatio         | 1.17     |
| EpisodesThisIter    | 6        |
| EpisodesSoFar       | 6        |
| TimestepsThisIter   | 6        |
| TimestepsSoFar      | 6        |
| TimeElapsedThisIter | 3.29     |
| TimeElapsed         | 3.29     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:55:21.899390: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', [Box(5,), Box(5,)], Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-55-18fT9imq'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 521      |
| GradNorm            | 64.2     |
| UpdateRatio         | 0.142    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 3.01     |
| TimeElapsed         | 3.01     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:55:25.517251: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', [Box(5,), Box(5,)], Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-55-22aswBvu'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 517      |
| GradNorm            | 25.7     |
| UpdateRatio         | 0.12     |
| EpisodesThisIter    | 12       |
| EpisodesSoFar       | 12       |
| TimestepsThisIter   | 12       |
| TimestepsSoFar      | 12       |
| TimeElapsedThisIter | 2.9      |
| TimeElapsed         | 2.9      |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:55:29.138161: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', [Box(5,), Box(5,)], Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-55-25YWkie8'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 517      |
| GradNorm            | 15.3     |
| UpdateRatio         | 0.119    |
| EpisodesThisIter    | 14       |
| EpisodesSoFar       | 14       |
| TimestepsThisIter   | 14       |
| TimestepsSoFar      | 14       |
| TimeElapsedThisIter | 2.91     |
| TimeElapsed         | 2.91     |
----------------------------------
Disconnecting client on fd 73
2018-02-02 23:55:32.757587: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', [Box(5,), Box(5,)], Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-55-29s2KNLG'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 292      |
| GradNorm            | 2.31e+04 |
| UpdateRatio         | 1.15     |
| EpisodesThisIter    | 4        |
| EpisodesSoFar       | 4        |
| TimestepsThisIter   | 4        |
| TimestepsSoFar      | 4        |
| TimeElapsedThisIter | 3.08     |
| TimeElapsed         | 3.08     |
----------------------------------
Disconnecting client on fd 74
2018-02-02 23:55:36.703615: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Tuple(Box(5,), Box(5,)), Tuple(Discrete(10), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-55-33CLcyWL'
Observation shape is ((10,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Discrete(10)
Observation shape is (10,)
Using one-hot preprocessor for discrete envs.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 25.7     |
| UpdateRatio         | 0.121    |
| EpisodesThisIter    | 12       |
| EpisodesSoFar       | 12       |
| TimestepsThisIter   | 12       |
| TimestepsSoFar      | 12       |
| TimeElapsedThisIter | 3.12     |
| TimeElapsed         | 3.12     |
----------------------------------
Disconnecting client on fd 74
2018-02-02 23:55:40.309733: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Tuple(Box(5,), Box(5,)), Discrete(5), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-55-37WdGw_Q'
Observation shape is (5,)
Using one-hot preprocessor for discrete envs.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 44.5     |
| UpdateRatio         | 0.119    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 2.92     |
| TimeElapsed         | 2.92     |
----------------------------------
Disconnecting client on fd 74
2018-02-02 23:55:44.244927: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Tuple(Box(5,), Box(5,)), Box(80, 80, 1), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-55-40Xfeg8w'
Observation shape is (80, 80, 1)
Not using any observation preprocessor.
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 288      |
| GradNorm            | 5.54e+03 |
| UpdateRatio         | 1.17     |
| EpisodesThisIter    | 6        |
| EpisodesSoFar       | 6        |
| TimestepsThisIter   | 6        |
| TimestepsSoFar      | 6        |
| TimeElapsedThisIter | 3.2      |
| TimeElapsed         | 3.2      |
----------------------------------
Disconnecting client on fd 74
2018-02-02 23:55:48.180877: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Tuple(Box(5,), Box(5,)), Box(128,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-55-4444BM2R'
Observation shape is (128,)
Assuming Atari ram env, using AtariRamPreprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 520      |
| GradNorm            | 64.5     |
| UpdateRatio         | 0.142    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 3.06     |
| TimeElapsed         | 3.06     |
----------------------------------
Disconnecting client on fd 74
2018-02-02 23:55:52.080641: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Tuple(Box(5,), Box(5,)), Tuple(Box(5,), Box(5,)), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-55-48RNlAto'
Observation shape is ((5,), (5,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(5,)
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 517      |
| GradNorm            | 45.4     |
| UpdateRatio         | 0.12     |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 3.1      |
| TimeElapsed         | 3.1      |
----------------------------------
Disconnecting client on fd 74
2018-02-02 23:55:55.626263: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Tuple(Box(5,), Box(5,)), Box(5,), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-55-521d3yOr'
Observation shape is (5,)
Not using any observation preprocessor.
Constructing fcnet [256, 256] <function tanh at 0x7fbbadecab90>
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 518      |
| GradNorm            | 44.7     |
| UpdateRatio         | 0.119    |
| EpisodesThisIter    | 10       |
| EpisodesSoFar       | 10       |
| TimestepsThisIter   | 10       |
| TimestepsSoFar      | 10       |
| TimeElapsedThisIter | 2.87     |
| TimeElapsed         | 2.87     |
----------------------------------
Disconnecting client on fd 74
2018-02-02 23:55:59.524961: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
ok
()
('=== Testing', 'ES', Tuple(Box(5,), Box(5,)), Box(210, 160, 3), '===')
Unified logger created with logdir '/root/ray_results/2018-02-02_23-55-55kBqxek'
Observation shape is (210, 160, 3)
Assuming Atari pixel env, using AtariPixelPreprocessor.
Creating shared noise table.
Creating actors.
Collected 0 episodes 0 timesteps so far this iter
----------------------------------
| EvalEpRewMean       | nan      |
| EvalEpRewStd        | nan      |
| EvalEpLenMean       | nan      |
| EpRewMean           | 1        |
| EpRewStd            | 0        |
| EpLenMean           | 1        |
| Norm                | 291      |
| GradNorm            | 2.31e+04 |
| UpdateRatio         | 1.15     |
| EpisodesThisIter    | 4        |
| EpisodesSoFar       | 4        |
| TimestepsThisIter   | 4        |
| TimestepsSoFar      | 4        |
| TimeElapsedThisIter | 3.16     |
| TimeElapsed         | 3.16     |
----------------------------------
ok
()
('A3C', 'action_space', 'discrete', 'obs_space', 'atari', 'result', 'ok')
('A3C', 'action_space', 'discrete', 'obs_space', 'atari_ram', 'result', 'ok')
('A3C', 'action_space', 'discrete', 'obs_space', 'discrete', 'result', 'ok')
('A3C', 'action_space', 'discrete', 'obs_space', 'image', 'result', 'ok')
('A3C', 'action_space', 'discrete', 'obs_space', 'mixed_tuple', 'result', 'ok')
('A3C', 'action_space', 'discrete', 'obs_space', 'simple_tuple', 'result', 'ok')
('A3C', 'action_space', 'discrete', 'obs_space', 'vector', 'result', 'ok')
('A3C', 'action_space', 'implicit_tuple', 'obs_space', 'atari', 'result', 'ERROR')
('A3C', 'action_space', 'implicit_tuple', 'obs_space', 'atari_ram', 'result', 'ERROR')
('A3C', 'action_space', 'implicit_tuple', 'obs_space', 'discrete', 'result', 'ERROR')
('A3C', 'action_space', 'implicit_tuple', 'obs_space', 'image', 'result', 'ERROR')
('A3C', 'action_space', 'implicit_tuple', 'obs_space', 'mixed_tuple', 'result', 'ERROR')
('A3C', 'action_space', 'implicit_tuple', 'obs_space', 'simple_tuple', 'result', 'ERROR')
('A3C', 'action_space', 'implicit_tuple', 'obs_space', 'vector', 'result', 'ERROR')
('A3C', 'action_space', 'simple_tuple', 'obs_space', 'atari', 'result', 'ERROR')
('A3C', 'action_space', 'simple_tuple', 'obs_space', 'atari_ram', 'result', 'ERROR')
('A3C', 'action_space', 'simple_tuple', 'obs_space', 'discrete', 'result', 'ERROR')
('A3C', 'action_space', 'simple_tuple', 'obs_space', 'image', 'result', 'ERROR')
('A3C', 'action_space', 'simple_tuple', 'obs_space', 'mixed_tuple', 'result', 'ERROR')
('A3C', 'action_space', 'simple_tuple', 'obs_space', 'simple_tuple', 'result', 'ERROR')
('A3C', 'action_space', 'simple_tuple', 'obs_space', 'vector', 'result', 'ERROR')
('A3C', 'action_space', 'vector', 'obs_space', 'atari', 'result', 'ok')
('A3C', 'action_space', 'vector', 'obs_space', 'atari_ram', 'result', 'ok')
('A3C', 'action_space', 'vector', 'obs_space', 'discrete', 'result', 'ok')
('A3C', 'action_space', 'vector', 'obs_space', 'image', 'result', 'ok')
('A3C', 'action_space', 'vector', 'obs_space', 'mixed_tuple', 'result', 'ok')
('A3C', 'action_space', 'vector', 'obs_space', 'simple_tuple', 'result', 'ok')
('A3C', 'action_space', 'vector', 'obs_space', 'vector', 'result', 'ok')
('DQN', 'action_space', 'discrete', 'obs_space', 'atari', 'result', 'ok')
('DQN', 'action_space', 'discrete', 'obs_space', 'atari_ram', 'result', 'ok')
('DQN', 'action_space', 'discrete', 'obs_space', 'discrete', 'result', 'ok')
('DQN', 'action_space', 'discrete', 'obs_space', 'image', 'result', 'ok')
('DQN', 'action_space', 'discrete', 'obs_space', 'mixed_tuple', 'result', 'ok')
('DQN', 'action_space', 'discrete', 'obs_space', 'simple_tuple', 'result', 'ok')
('DQN', 'action_space', 'discrete', 'obs_space', 'vector', 'result', 'ok')
('DQN', 'action_space', 'implicit_tuple', 'obs_space', 'atari', 'result', 'unsupported')
('DQN', 'action_space', 'implicit_tuple', 'obs_space', 'atari_ram', 'result', 'unsupported')
('DQN', 'action_space', 'implicit_tuple', 'obs_space', 'discrete', 'result', 'unsupported')
('DQN', 'action_space', 'implicit_tuple', 'obs_space', 'image', 'result', 'unsupported')
('DQN', 'action_space', 'implicit_tuple', 'obs_space', 'mixed_tuple', 'result', 'unsupported')
('DQN', 'action_space', 'implicit_tuple', 'obs_space', 'simple_tuple', 'result', 'unsupported')
('DQN', 'action_space', 'implicit_tuple', 'obs_space', 'vector', 'result', 'unsupported')
('DQN', 'action_space', 'simple_tuple', 'obs_space', 'atari', 'result', 'unsupported')
('DQN', 'action_space', 'simple_tuple', 'obs_space', 'atari_ram', 'result', 'unsupported')
('DQN', 'action_space', 'simple_tuple', 'obs_space', 'discrete', 'result', 'unsupported')
('DQN', 'action_space', 'simple_tuple', 'obs_space', 'image', 'result', 'unsupported')
('DQN', 'action_space', 'simple_tuple', 'obs_space', 'mixed_tuple', 'result', 'unsupported')
('DQN', 'action_space', 'simple_tuple', 'obs_space', 'simple_tuple', 'result', 'unsupported')
('DQN', 'action_space', 'simple_tuple', 'obs_space', 'vector', 'result', 'unsupported')
('DQN', 'action_space', 'vector', 'obs_space', 'atari', 'result', 'unsupported')
('DQN', 'action_space', 'vector', 'obs_space', 'atari_ram', 'result', 'unsupported')
('DQN', 'action_space', 'vector', 'obs_space', 'discrete', 'result', 'unsupported')
('DQN', 'action_space', 'vector', 'obs_space', 'image', 'result', 'unsupported')
('DQN', 'action_space', 'vector', 'obs_space', 'mixed_tuple', 'result', 'unsupported')
('DQN', 'action_space', 'vector', 'obs_space', 'simple_tuple', 'result', 'unsupported')
('DQN', 'action_space', 'vector', 'obs_space', 'vector', 'result', 'unsupported')
('ES', 'action_space', 'discrete', 'obs_space', 'atari', 'result', 'ok')
('ES', 'action_space', 'discrete', 'obs_space', 'atari_ram', 'result', 'ok')
('ES', 'action_space', 'discrete', 'obs_space', 'discrete', 'result', 'ok')
('ES', 'action_space', 'discrete', 'obs_space', 'image', 'result', 'ok')
('ES', 'action_space', 'discrete', 'obs_space', 'mixed_tuple', 'result', 'ok')
('ES', 'action_space', 'discrete', 'obs_space', 'simple_tuple', 'result', 'ok')
('ES', 'action_space', 'discrete', 'obs_space', 'vector', 'result', 'ok')
('ES', 'action_space', 'implicit_tuple', 'obs_space', 'atari', 'result', 'ok')
('ES', 'action_space', 'implicit_tuple', 'obs_space', 'atari_ram', 'result', 'ok')
('ES', 'action_space', 'implicit_tuple', 'obs_space', 'discrete', 'result', 'ok')
('ES', 'action_space', 'implicit_tuple', 'obs_space', 'image', 'result', 'ok')
('ES', 'action_space', 'implicit_tuple', 'obs_space', 'mixed_tuple', 'result', 'ok')
('ES', 'action_space', 'implicit_tuple', 'obs_space', 'simple_tuple', 'result', 'ok')
('ES', 'action_space', 'implicit_tuple', 'obs_space', 'vector', 'result', 'ok')
('ES', 'action_space', 'simple_tuple', 'obs_space', 'atari', 'result', 'ok')
('ES', 'action_space', 'simple_tuple', 'obs_space', 'atari_ram', 'result', 'ok')
('ES', 'action_space', 'simple_tuple', 'obs_space', 'discrete', 'result', 'ok')
('ES', 'action_space', 'simple_tuple', 'obs_space', 'image', 'result', 'ok')
('ES', 'action_space', 'simple_tuple', 'obs_space', 'mixed_tuple', 'result', 'ok')
('ES', 'action_space', 'simple_tuple', 'obs_space', 'simple_tuple', 'result', 'ok')
('ES', 'action_space', 'simple_tuple', 'obs_space', 'vector', 'result', 'ok')
('ES', 'action_space', 'vector', 'obs_space', 'atari', 'result', 'ok')
('ES', 'action_space', 'vector', 'obs_space', 'atari_ram', 'result', 'ok')
('ES', 'action_space', 'vector', 'obs_space', 'discrete', 'result', 'ok')
('ES', 'action_space', 'vector', 'obs_space', 'image', 'result', 'ok')
('ES', 'action_space', 'vector', 'obs_space', 'mixed_tuple', 'result', 'ok')
('ES', 'action_space', 'vector', 'obs_space', 'simple_tuple', 'result', 'ok')
('ES', 'action_space', 'vector', 'obs_space', 'vector', 'result', 'ok')
('PPO', 'action_space', 'discrete', 'obs_space', 'atari', 'result', 'ok')
('PPO', 'action_space', 'discrete', 'obs_space', 'atari_ram', 'result', 'ok')
('PPO', 'action_space', 'discrete', 'obs_space', 'discrete', 'result', 'ok')
('PPO', 'action_space', 'discrete', 'obs_space', 'image', 'result', 'ok')
('PPO', 'action_space', 'discrete', 'obs_space', 'mixed_tuple', 'result', 'ok')
('PPO', 'action_space', 'discrete', 'obs_space', 'simple_tuple', 'result', 'ok')
('PPO', 'action_space', 'discrete', 'obs_space', 'vector', 'result', 'ok')
('PPO', 'action_space', 'implicit_tuple', 'obs_space', 'atari', 'result', 'ok')
('PPO', 'action_space', 'implicit_tuple', 'obs_space', 'atari_ram', 'result', 'ok')
('PPO', 'action_space', 'implicit_tuple', 'obs_space', 'discrete', 'result', 'ok')
('PPO', 'action_space', 'implicit_tuple', 'obs_space', 'image', 'result', 'ok')
('PPO', 'action_space', 'implicit_tuple', 'obs_space', 'mixed_tuple', 'result', 'ok')
('PPO', 'action_space', 'implicit_tuple', 'obs_space', 'simple_tuple', 'result', 'ok')
('PPO', 'action_space', 'implicit_tuple', 'obs_space', 'vector', 'result', 'ok')
('PPO', 'action_space', 'simple_tuple', 'obs_space', 'atari', 'result', 'ok')
('PPO', 'action_space', 'simple_tuple', 'obs_space', 'atari_ram', 'result', 'ok')
('PPO', 'action_space', 'simple_tuple', 'obs_space', 'discrete', 'rok

----------------------------------------------------------------------
Ran 1 test in 345.388s

OK
esult', 'ok')
('PPO', 'action_space', 'simple_tuple', 'obs_space', 'image', 'result', 'ok')
('PPO', 'action_space', 'simple_tuple', 'obs_space', 'mixed_tuple', 'result', 'ok')
('PPO', 'action_space', 'simple_tuple', 'obs_space', 'simple_tuple', 'result', 'ok')
('PPO', 'action_space', 'simple_tuple', 'obs_space', 'vector', 'result', 'ok')
('PPO', 'action_space', 'vector', 'obs_space', 'atari', 'result', 'ok')
('PPO', 'action_space', 'vector', 'obs_space', 'atari_ram', 'result', 'ok')
('PPO', 'action_space', 'vector', 'obs_space', 'discrete', 'result', 'ok')
('PPO', 'action_space', 'vector', 'obs_space', 'image', 'result', 'ok')
('PPO', 'action_space', 'vector', 'obs_space', 'mixed_tuple', 'result', 'ok')
('PPO', 'action_space', 'vector', 'obs_space', 'simple_tuple', 'result', 'ok')
('PPO', 'action_space', 'vector', 'obs_space', 'vector', 'result', 'ok')
Disconnecting client on fd 11
Disconnecting client on fd 74
Disconnecting client on fd 61
Disconnecting client on fd 60
Disconnecting client on fd 59
Disconnecting client on fd 58
Disconnecting client on fd 57
[WARN] (/ray/src/global_scheduler/global_scheduler.cc:412) Missed too many heartbeats from local scheduler, marking as dead.
Disconnecting client on fd 56
Disconnecting client on fd 55
Disconnecting client on fd 54
Disconnecting client on fd 53
Disconnecting client on fd 52
Disconnecting client on fd 51
Disconnecting client on fd 50
Disconnecting client on fd 49
Disconnecting client on fd 48
Disconnecting client on fd 47
Disconnecting client on fd 46
Disconnecting client on fd 45
Disconnecting client on fd 44
Disconnecting client on fd 43
Disconnecting client on fd 42
Disconnecting client on fd 41
Disconnecting client on fd 40
Disconnecting client on fd 39
Disconnecting client on fd 38
Disconnecting client on fd 37
Disconnecting client on fd 36
Disconnecting client on fd 35
Disconnecting client on fd 34
Disconnecting client on fd 30
Disconnecting client on fd 33
Disconnecting client on fd 31
Disconnecting client on fd 32
Disconnecting client on fd 29
Disconnecting client on fd 28
Disconnecting client on fd 27
Disconnecting client on fd 26
Disconnecting client on fd 25
Disconnecting client on fd 24
Disconnecting client on fd 23
Disconnecting client on fd 22
Disconnecting client on fd 21
Disconnecting client on fd 20
Disconnecting client on fd 19
Disconnecting client on fd 18
Disconnecting client on fd 17
Disconnecting client on fd 16
Disconnecting client on fd 15
Disconnecting client on fd 14
Disconnecting client on fd 9
Disconnecting client on fd 7
+ docker run --rm --shm-size=10G --memory=10G f92f28c594d96220fa3e04f4e1c18a53a5eec4680ab2f9ab421ae5f4543a7978 python /ray/python/ray/tune/examples/tune_mnist_ray.py --fast
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
Allowing the Plasma store to use up to 8.58993GB of memory.
Starting object store with directory /dev/shm and huge page support disabled
2018-02-02 23:56:33.906284: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Disconnecting client on fd 59
2018-02-02 23:56:38.935689: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-02-02 23:56:38.975402: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Waiting for redis server at 127.0.0.1:33460 to respond...
Waiting for redis server at 127.0.0.1:44311 to respond...
Warning: Reducing object store memory because /dev/shm has only 10737418240 bytes available. You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you may need to pass an argument with the flag '--shm-size' to 'docker run'.
Starting local scheduler with the following resources: {'GPU': 0, 'CPU': 48}.

======================================================================
View the web UI at http://localhost:8888/notebooks/ray_ui32345.ipynb?token=3b6d48cd77f1eb5b80cbf5be3a8739e612ecd9d615778e09
======================================================================

== Status ==
Using FIFO scheduling algorithm.
Result logdir: /root/ray_results/tune_mnist_test
PENDING trials:
 - train_mnist_0_activation=relu:	PENDING
 - train_mnist_1_activation=elu:	PENDING
 - train_mnist_2_activation=tanh:	PENDING

Unified logger created with logdir '/root/ray_results/tune_mnist_test/train_mnist_0_activation=relu_2018-02-02_23-56-07w06FGN'
WARNING: Serializing objects of type <class 'ray.tune.registry._Registry'> by expanding them as dictionaries of their fields. This behavior may be incorrect in some cases.
== Status ==
Using FIFO scheduling algorithm.
Resources used: 1/48 CPUs, 0/0 GPUs
Result logdir: /root/ray_results/tune_mnist_test
PENDING trials:
 - train_mnist_1_activation=elu:	PENDING
 - train_mnist_2_activation=tanh:	PENDING
RUNNING trials:
 - train_mnist_0_activation=relu:	RUNNING

Unified logger created with logdir '/root/ray_results/tune_mnist_test/train_mnist_1_activation=elu_2018-02-02_23-56-07txfPfq'
Unified logger created with logdir '/root/ray_results/tune_mnist_test/train_mnist_2_activation=tanh_2018-02-02_23-56-07JVJNdx'
TrainingResult for train_mnist_2_activation=tanh:
  date: 2018-02-02_23-56-34
  experiment_id: 88faa89292d94cccb552125186df92c8
  hostname: 2f0a12d18f6e
  mean_accuracy: 0.019999999552965164
  pid: !!python/long '92'
  time_this_iter_s: 2.0022761821746826
  time_total_s: 2.0022761821746826
  timestamp: !!python/long '1517615794'
  timesteps_this_iter: !!python/long '0'
  timesteps_total: !!python/long '0'
  training_iteration: !!python/long '1'
  
== Status ==
Using FIFO scheduling algorithm.
Resources used: 3/48 CPUs, 0/0 GPUs
Result logdir: /root/ray_results/tune_mnist_test
RUNNING trials:
 - train_mnist_0_activation=relu:	RUNNING
 - train_mnist_1_activation=elu:	RUNNING
 - train_mnist_2_activation=tanh:	RUNNING [pid=92], 2 s, 0 ts, 0.02 acc

TrainingResult for train_mnist_2_activation=tanh:
  date: 2018-02-02_23-56-35
  done: true
  experiment_id: 88faa89292d94cccb552125186df92c8
  hostname: 2f0a12d18f6e
  mean_accuracy: 0.6399999856948853
  pid: !!python/long '92'
  time_this_iter_s: 1.001086950302124
  time_total_s: 3.0033631324768066
  timestamp: !!python/long '1517615795'
  timesteps_this_iter: !!python/long '30'
  timesteps_total: !!python/long '30'
  training_iteration: !!python/long '2'
  
Stopping train_mnist_2_activation=tanh Actor timed out, but moving on...
TrainingResult for train_mnist_0_activation=relu:
  date: 2018-02-02_23-56-39
  experiment_id: 6fe060a9eb664beeaf3b8e00a3c9ec37
  hostname: 2f0a12d18f6e
  mean_accuracy: 0.11999999731779099
  pid: !!python/long '88'
  time_this_iter_s: 7.007778882980347
  time_total_s: 7.007778882980347
  timestamp: !!python/long '1517615799'
  timesteps_this_iter: !!python/long '0'
  timesteps_total: !!python/long '0'
  training_iteration: !!python/long '1'
  
TrainingResult for train_mnist_1_activation=elu:
  date: 2018-02-02_23-56-39
  experiment_id: dcef1bd6f4944a26affed8ddd2115ac5
  hostname: 2f0a12d18f6e
  mean_accuracy: 0.05999999865889549
  pid: !!python/long '90'
  time_this_iter_s: 7.007719993591309
  time_total_s: 7.007719993591309
  timestamp: !!python/long '1517615799'
  timesteps_this_iter: !!python/long '0'
  timesteps_total: !!python/long '0'
  training_iteration: !!python/long '1'
  
== Status ==
Using FIFO scheduling algorithm.
Resources used: 2/48 CPUs, 0/0 GPUs
Result logdirDisconnecting client on fd 60
Disconnecting client on fd 63
Disconnecting client on fd 11
Disconnecting client on fd 62
[WARN] (/ray/src/global_scheduler/global_scheduler.cc:412) Missed too many heartbeats from local scheduler, marking as dead.
Disconnecting client on fd 61
Disconnecting client on fd 58
Disconnecting client on fd 57
Disconnecting client on fd 56
Disconnecting client on fd 55
Disconnecting client on fd 54
Disconnecting client on fd 53
Disconnecting client on fd 52
Disconnecting client on fd 51
Disconnecting client on fd 50
Disconnecting client on fd 49
Disconnecting client on fd 48
Disconnecting client on fd 47
Disconnecting client on fd 46
Disconnecting client on fd 45
Disconnecting client on fd 44
Disconnecting client on fd 43
Disconnecting client on fd 42
Disconnecting client on fd 41
Disconnecting client on fd 40
Disconnecting client on fd 39
Disconnecting client on fd 38
Disconnecting client on fd 37
Disconnecting client on fd 36
Disconnecting client on fd 35
Disconnecting client on fd 34
Disconnecting client on fd 33
Disconnecting client on fd 32
Disconnecting client on fd 31
Disconnecting client on fd 30
Disconnecting client on fd 29
Disconnecting client on fd 28
Disconnecting client on fd 27
Disconnecting client on fd 26
Disconnecting client on fd 25
Disconnecting client on fd 24
Disconnecting client on fd 23
Disconnecting client on fd 22
Disconnecting client on fd 21
Disconnecting client on fd 20
Disconnecting client on fd 19
Disconnecting client on fd 18
Disconnecting client on fd 17
Disconnecting client on fd 16
Disconnecting client on fd 15
Disconnecting client on fd 14
Disconnecting client on fd 13
Disconnecting client on fd 9
Disconnecting client on fd 7
: /root/ray_results/tune_mnist_test
RUNNING trials:
 - train_mnist_0_activation=relu:	RUNNING [pid=88], 7 s, 0 ts, 0.12 acc
 - train_mnist_1_activation=elu:	RUNNING [pid=90], 7 s, 0 ts, 0.06 acc
TERMINATED trials:
 - train_mnist_2_activation=tanh:	TERMINATED [pid=92], 3 s, 30 ts, 0.64 acc

TrainingResult for train_mnist_0_activation=relu:
  date: 2018-02-02_23-56-40
  done: true
  experiment_id: 6fe060a9eb664beeaf3b8e00a3c9ec37
  hostname: 2f0a12d18f6e
  mean_accuracy: 0.46000000834465027
  pid: !!python/long '88'
  time_this_iter_s: 1.0011870861053467
  time_total_s: 8.008965969085693
  timestamp: !!python/long '1517615800'
  timesteps_this_iter: !!python/long '20'
  timesteps_total: !!python/long '20'
  training_iteration: !!python/long '2'
  
Stopping train_mnist_0_activation=relu Actor timed out, but moving on...
TrainingResult for train_mnist_1_activation=elu:
  date: 2018-02-02_23-56-40
  done: true
  experiment_id: dcef1bd6f4944a26affed8ddd2115ac5
  hostname: 2f0a12d18f6e
  mean_accuracy: 0.5
  pid: !!python/long '90'
  time_this_iter_s: 1.0010981559753418
  time_total_s: 8.00881814956665
  timestamp: !!python/long '1517615800'
  timesteps_this_iter: !!python/long '20'
  timesteps_total: !!python/long '20'
  training_iteration: !!python/long '2'
  
Stopping train_mnist_1_activation=elu Actor timed out, but moving on...
== Status ==
Using FIFO scheduling algorithm.
Resources used: 0/48 CPUs, 0/0 GPUs
Result logdir: /root/ray_results/tune_mnist_test
TERMINATED trials:
 - train_mnist_0_activation=relu:	TERMINATED [pid=88], 8 s, 20 ts, 0.46 acc
 - train_mnist_1_activation=elu:	TERMINATED [pid=90], 8 s, 20 ts, 0.5 acc
 - train_mnist_2_activation=tanh:	TERMINATED [pid=92], 3 s, 30 ts, 0.64 acc

+ docker run --rm --shm-size=10G --memory=10G f92f28c594d96220fa3e04f4e1c18a53a5eec4680ab2f9ab421ae5f4543a7978 python /ray/python/ray/rllib/examples/multiagent_mountaincar.py
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
[2018-02-02 23:56:46,742] Making new env: MultiAgentMountainCarEnv-v0
2018-02-02 23:56:46.757488: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Waiting for redis server at 127.0.0.1:43569 to respond...
Waiting for redis server at 127.0.0.1:46720 to respond...
Warning: Reducing object store memory because /dev/shm has only 10737418240 bytes available. You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you may need to pass an argument with the flag '--shm-size' to 'docker run'.
Starting local scheduler with the following resources: {'GPU': 0, 'CPU': 4}.

======================================================================
View the web UI at http://localhost:8888/notebooks/ray_ui74096.ipynb?token=381f2bf36f09bf44cc3724d9d74c7715e74198a06da35f6b
======================================================================

Unified logger created with logdir '/root/ray_results/2018-02-02_23-56-46uLKOlE'
Observation shape is ((2,), (2,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(2,)
Observation shape is (2,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(2,)
Observation shape is (2,)
Not using any observation preprocessor.
Constructing fcnet [32, 32] <function tanh at 0x7f21ed93f668>
Constructing fcnet [32, 32] <function tanh at 0x7f21ed93f668>
Constructing fcnet [32, 32] <function tanh at 0x7f21ed93f668>
Constructing fcnet [32, 32] <function tanh at 0x7f21ed93f668>
Constructing fcnet [32, 32] <function tanh at 0x7f21ed93f668>
Constructing fcnet [32, 32] <function tanh at 0x7f21ed93f668>
Constructing fcnet [32, 32] <function tanh at 0x7f21ed93f668>
Constructing fcnet [32, 32] <function tanh at 0x7f21ed93f668>
Constructing fcnet [32, 32] <function tanh at 0x7f21ed93f668>
Constructing fcnet [32, 32] <function tanh at 0x7f21ed93f668>
WARNING: Serializing objects of type <class 'ray.tune.registry._Registry'> by expanding them as dictionaries of their fields. This behavior may be incorrect in some cases.
===> iteration 0
Computing policy (iterations=10, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.35514e-03    1.35492e-03    0.00000e+00    1.09053e-06    2.19722e+00
              1    1.09246e-03    1.09121e-03    0.00000e+00    6.23321e-06    2.19721e+00
              2    8.47699e-04    8.44457e-04    0.00000e+00    1.62193e-05    2.19720e+00
              3    6.31453e-04    6.24932e-04    0.00000e+00    3.26410e-05    2.19719e+00
              4    3.97740e-04    3.86475e-04    0.00000e+00    5.63556e-05    2.19716e+00
              5    1.84685e-04    1.67355e-04    0.00000e+00    8.66731e-05    2.19713e+00
              6   -8.52346e-05   -1.09824e-04    0.00000e+00    1.22945e-04    2.19710e+00
              7   -2.86735e-04   -3.19484e-04    0.00000e+00    1.63745e-04    2.19705e+00
              8   -5.46741e-04   -5.88865e-04    0.00000e+00    2.10632e-04    2.19701e+00
              9   -7.19592e-04   -7.73115e-04    0.00000e+00    2.67633e-04    2.19695e+00
+ docker run --rm --shm-size=10G --memory=10G f92f28c594d96220fa3e04f4e1c18a53a5eec4680ab2f9ab421ae5f4543a7978 python /ray/python/ray/rllib/examples/multiagent_pendulum.py
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
[2018-02-02 23:57:03,450] Making new env: MultiAgentPendulumEnv-v0
2018-02-02 23:57:03.474849: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Waiting for redis server at 127.0.0.1:19780 to respond...
Waiting for redis server at 127.0.0.1:13622 to respond...
Warning: Reducing object store memory because /dev/shm has only 10737418240 bytes available. You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you may need to pass an argument with the flag '--shm-size' to 'docker run'.
Starting local scheduler with the following resources: {'GPU': 0, 'CPU': 4}.

======================================================================
View the web UI at http://localhost:8888/notebooks/ray_ui36158.ipynb?token=4cd46842b7c8d9551143619418683a75a567fa5dd0476a4c
======================================================================

Unified logger created with logdir '/root/ray_results/2018-02-02_23-57-03RyKd4u'
Observation shape is ((3,), (3,))
Using a TupleFlatteningPreprocessor
Creating sub-preprocessor for Box(3,)
Observation shape is (3,)
Not using any observation preprocessor.
Creating sub-preprocessor for Box(3,)
Observation shape is (3,)
Not using any observation preprocessor.
Constructing fcnet [32, 32] <function tanh at 0x7fa42baff668>
Constructing fcnet [32, 32] <function tanh at 0x7fa42baff668>
Constructing fcnet [256, 256] <function tanh at 0x7fa42baff668>
Constructing fcnet [32, 32] <function tanh at 0x7fa42baff668>
Constructing fcnet [32, 32] <function tanh at 0x7fa42baff668>
Constructing fcnet [256, 256] <function tanh at 0x7fa42baff668>
Constructing fcnet [32, 32] <function tanh at 0x7fa42baff668>
Constructing fcnet [32, 32] <function tanh at 0x7fa42baff668>
Constructing fcnet [256, 256] <function tanh at 0x7fa42baff668>
Constructing fcnet [32, 32] <function tanh at 0x7fa42baff668>
Constructing fcnet [32, 32] <function tanh at 0x7fa42baff668>
Constructing fcnet [256, 256] <function tanh at 0x7fa42baff668>
Constructing fcnet [32, 32] <function tanh at 0x7fa42baff668>
Constructing fcnet [32, 32] <function tanh at 0x7fa42baff668>
Constructing fcnet [256, 256] <function tanh at 0x7fa42baff668>
WARNING: Serializing objects of type <class 'ray.tune.registry._Registry'> by expanding them as dictionaries of their fields. This behavior may be incorrect in some cases.
===> iteration 0
Computing policy (iterations=10, stepsize=5e-05):
           iter     total loss    policy loss        vf loss             kl        entropy
              0    1.76511e+03    5.78051e-03    1.76510e+03    3.40448e-06    2.83904e+00
              1    1.76449e+03    5.30818e-03    1.76449e+03    1.31330e-05    2.83925e+00
              2    1.76381e+03    4.97845e-03    1.76380e+03    3.18568e-05    2.83947e+00
              3    1.76311e+03    4.63741e-03    1.76311e+03    5.77574e-05    2.83963e+00
              4    1.76222e+03    4.32059e-03    1.76222e+03    9.69446e-05    2.83982e+00
              5    1.76121e+03    3.92106e-03    1.76121e+03    1.43252e-04    2.83999e+00
              6    1.76006e+03    3.59652e-03    1.76006e+03    1.93913e-04    2.84014e+00
              7    1.75867e+03    3.28705e-03    1.75866e+03    2.57149e-04    2.84035e+00
              8    1.75688e+03    2.92687e-03    1.75688e+03    3.30161e-04    2.84052e+00
              9    1.75458e+03    2.57681e-03    1.75457e+03    4.06818e-04    2.84071e+00
Notifying endpoint 'HTTP:'
ERROR: Failed to notify endpoint 'HTTP:'
java.net.MalformedURLException: no protocol: 
	at java.net.URL.<init>(URL.java:593)
	at java.net.URL.<init>(URL.java:490)
	at java.net.URL.<init>(URL.java:439)
	at com.tikal.hudson.plugins.notification.Protocol$3.send(Protocol.java:61)
	at com.tikal.hudson.plugins.notification.Phase.handle(Phase.java:45)
	at com.tikal.hudson.plugins.notification.JobListener.onCompleted(JobListener.java:36)
	at hudson.model.listeners.RunListener.fireCompleted(RunListener.java:201)
	at hudson.model.Run.execute(Run.java:1783)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:410)
Failed to notify endpoint 'HTTP:' - java.net.MalformedURLException: no protocol: 
Finished: SUCCESS