20/08/03 12:50:30.334 pool-1-thread-1 INFO SparkContext: Running Spark version 3.0.1-SNAPSHOT 20/08/03 12:50:30.606 pool-1-thread-1 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 20/08/03 12:50:30.711 pool-1-thread-1 INFO ResourceUtils: ============================================================== 20/08/03 12:50:30.712 pool-1-thread-1 INFO ResourceUtils: Resources for spark.driver: 20/08/03 12:50:30.712 pool-1-thread-1 INFO ResourceUtils: ============================================================== 20/08/03 12:50:30.713 pool-1-thread-1 INFO SparkContext: Submitted application: MLlibUnitTest 20/08/03 12:50:30.792 pool-1-thread-1 INFO SecurityManager: Changing view acls to: jenkins 20/08/03 12:50:30.792 pool-1-thread-1 INFO SecurityManager: Changing modify acls to: jenkins 20/08/03 12:50:30.793 pool-1-thread-1 INFO SecurityManager: Changing view acls groups to: 20/08/03 12:50:30.793 pool-1-thread-1 INFO SecurityManager: Changing modify acls groups to: 20/08/03 12:50:30.794 pool-1-thread-1 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(jenkins); groups with view permissions: Set(); users with modify permissions: Set(jenkins); groups with modify permissions: Set() 20/08/03 12:50:31.921 pool-1-thread-1 INFO Utils: Successfully started service 'sparkDriver' on port 45359. 20/08/03 12:50:31.955 pool-1-thread-1 INFO SparkEnv: Registering MapOutputTracker 20/08/03 12:50:31.987 pool-1-thread-1 INFO SparkEnv: Registering BlockManagerMaster 20/08/03 12:50:31.991 pool-1-thread-1 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information 20/08/03 12:50:31.991 pool-1-thread-1 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up 20/08/03 12:50:31.995 pool-1-thread-1 INFO SparkEnv: Registering BlockManagerMasterHeartbeat 20/08/03 12:50:32.010 pool-1-thread-1 INFO DiskBlockManager: Created local directory at /home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/blockmgr-757fbc3f-45bc-4c0c-ace1-1c3f9b5802f8 20/08/03 12:50:32.038 pool-1-thread-1 INFO MemoryStore: MemoryStore started with capacity 2.1 GiB 20/08/03 12:50:32.060 pool-1-thread-1 INFO SparkEnv: Registering OutputCommitCoordinator 20/08/03 12:50:32.218 pool-1-thread-1 INFO Executor: Starting executor ID driver on host amp-jenkins-worker-06.amp 20/08/03 12:50:32.244 pool-1-thread-1 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 34426. 20/08/03 12:50:32.244 pool-1-thread-1 INFO NettyBlockTransferService: Server created on amp-jenkins-worker-06.amp:34426 20/08/03 12:50:32.245 pool-1-thread-1 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy 20/08/03 12:50:32.251 pool-1-thread-1 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, amp-jenkins-worker-06.amp, 34426, None) 20/08/03 12:50:32.255 dispatcher-BlockManagerMaster INFO BlockManagerMasterEndpoint: Registering block manager amp-jenkins-worker-06.amp:34426 with 2.1 GiB RAM, BlockManagerId(driver, amp-jenkins-worker-06.amp, 34426, None) 20/08/03 12:50:32.258 pool-1-thread-1 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, amp-jenkins-worker-06.amp, 34426, None) 20/08/03 12:50:32.259 pool-1-thread-1 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, amp-jenkins-worker-06.amp, 34426, None) 20/08/03 12:50:32.451 pool-1-thread-1 INFO log: Logging initialized @5105ms to org.eclipse.jetty.util.log.Slf4jLog 20/08/03 12:50:33.178 pool-1-thread-1 INFO SharedState: loading hive config file: file:/home/jenkins/workspace/NewSparkPullRequestBuilder/sql/core/target/scala-2.12/test-classes/hive-site.xml 20/08/03 12:50:33.213 pool-1-thread-1 INFO SharedState: Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/spark-warehouse'). 20/08/03 12:50:33.214 pool-1-thread-1 INFO SharedState: Warehouse path is 'file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/spark-warehouse'. 20/08/03 12:50:34.194 pool-1-thread-1 INFO CodeGenerator: Code generated in 268.832916 ms 20/08/03 12:50:35.633 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: ===== TEST OUTPUT FOR o.a.s.ml.classification.LogisticRegressionSuite: 'params' ===== 20/08/03 12:50:36.244 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: ===== FINISHED o.a.s.ml.classification.LogisticRegressionSuite: 'params' ===== 20/08/03 12:50:36.261 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: ===== TEST OUTPUT FOR o.a.s.ml.classification.LogisticRegressionSuite: 'logistic regression: default params' ===== 20/08/03 12:50:36.380 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 23.295654 ms 20/08/03 12:50:36.466 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [ccf2a6df] Stage class: LogisticRegression 20/08/03 12:50:36.466 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [ccf2a6df] Stage uid: logreg_77ca575705f3 20/08/03 12:50:36.516 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 17.777094 ms 20/08/03 12:50:36.537 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [ccf2a6df] training: numPartitions=2 storageLevel=StorageLevel(1 replicas) 20/08/03 12:50:36.557 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [ccf2a6df] {} 20/08/03 12:50:36.652 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at LogisticRegression.scala:499 20/08/03 12:50:36.674 dag-scheduler-event-loop INFO DAGScheduler: Got job 0 (treeAggregate at LogisticRegression.scala:499) with 2 output partitions 20/08/03 12:50:36.674 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 0 (treeAggregate at LogisticRegression.scala:499) 20/08/03 12:50:36.675 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:36.682 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:36.687 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[15] at treeAggregate at LogisticRegression.scala:499), which has no missing parents 20/08/03 12:50:36.793 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 12.6 KiB, free 2.1 GiB) 20/08/03 12:50:36.848 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 6.4 KiB, free 2.1 GiB) 20/08/03 12:50:36.850 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 6.4 KiB, free: 2.1 GiB) 20/08/03 12:50:36.852 dag-scheduler-event-loop INFO SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:36.868 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 0 (MapPartitionsRDD[15] at treeAggregate at LogisticRegression.scala:499) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:36.869 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 0.0 with 2 tasks 20/08/03 12:50:36.927 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:36.931 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:36.938 Executor task launch worker for task 1 INFO Executor: Running task 1.0 in stage 0.0 (TID 1) 20/08/03 12:50:36.938 Executor task launch worker for task 0 INFO Executor: Running task 0.0 in stage 0.0 (TID 0) 20/08/03 12:50:37.555 Executor task launch worker for task 0 INFO CodeGenerator: Code generated in 18.696478 ms 20/08/03 12:50:37.575 Executor task launch worker for task 1 INFO MemoryStore: Block rdd_9_1 stored as values in memory (estimated size 3.7 KiB, free 2.1 GiB) 20/08/03 12:50:37.576 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_9_1 in memory on amp-jenkins-worker-06.amp:34426 (size: 3.7 KiB, free: 2.1 GiB) 20/08/03 12:50:37.578 Executor task launch worker for task 0 INFO MemoryStore: Block rdd_9_0 stored as values in memory (estimated size 3.7 KiB, free 2.1 GiB) 20/08/03 12:50:37.579 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_9_0 in memory on amp-jenkins-worker-06.amp:34426 (size: 3.7 KiB, free: 2.1 GiB) 20/08/03 12:50:37.604 Executor task launch worker for task 1 INFO Executor: Finished task 1.0 in stage 0.0 (TID 1). 2629 bytes result sent to driver 20/08/03 12:50:37.604 Executor task launch worker for task 0 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 2629 bytes result sent to driver 20/08/03 12:50:37.613 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 682 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:37.615 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 702 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:37.616 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool 20/08/03 12:50:37.624 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 0 (treeAggregate at LogisticRegression.scala:499) finished in 0.924 s 20/08/03 12:50:37.629 dag-scheduler-event-loop INFO DAGScheduler: Job 0 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:37.630 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 0: Stage finished 20/08/03 12:50:37.633 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 0 finished: treeAggregate at LogisticRegression.scala:499, took 0.980473 s 20/08/03 12:50:37.643 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [ccf2a6df] {"numExamples":100} 20/08/03 12:50:37.645 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [ccf2a6df] {"lowestLabelWeight":"34.0"} 20/08/03 12:50:37.645 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [ccf2a6df] {"highestLabelWeight":"66.0"} 20/08/03 12:50:37.648 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [ccf2a6df] {"sumOfWeights":100.0} 20/08/03 12:50:37.653 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [ccf2a6df] {"numClasses":2} 20/08/03 12:50:37.653 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [ccf2a6df] {"numFeatures":1} 20/08/03 12:50:37.655 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 48.0 B, free 2.1 GiB) 20/08/03 12:50:37.658 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 77.0 B, free 2.1 GiB) 20/08/03 12:50:37.659 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 77.0 B, free: 2.1 GiB) 20/08/03 12:50:37.660 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 1 from broadcast at LogisticRegression.scala:593 20/08/03 12:50:38.088 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:38.093 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 153.0 B, free 2.1 GiB) 20/08/03 12:50:38.094 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 153.0 B, free: 2.1 GiB) 20/08/03 12:50:38.095 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 2 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:38.128 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:38.130 dag-scheduler-event-loop INFO DAGScheduler: Got job 1 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:38.130 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 1 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:38.130 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:38.133 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:38.136 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 1 (MapPartitionsRDD[16] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:38.143 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:38.148 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:38.149 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:38.150 dag-scheduler-event-loop INFO SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:38.151 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 1 (MapPartitionsRDD[16] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:38.152 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 1.0 with 2 tasks 20/08/03 12:50:38.157 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 2, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:38.159 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 1.0 (TID 3, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:38.159 Executor task launch worker for task 3 INFO Executor: Running task 1.0 in stage 1.0 (TID 3) 20/08/03 12:50:38.159 Executor task launch worker for task 2 INFO Executor: Running task 0.0 in stage 1.0 (TID 2) 20/08/03 12:50:38.176 Executor task launch worker for task 2 INFO BlockManager: Found block rdd_9_0 locally 20/08/03 12:50:38.177 Executor task launch worker for task 3 INFO BlockManager: Found block rdd_9_1 locally 20/08/03 12:50:38.215 Executor task launch worker for task 2 INFO Executor: Finished task 0.0 in stage 1.0 (TID 2). 2211 bytes result sent to driver 20/08/03 12:50:38.215 Executor task launch worker for task 3 INFO Executor: Finished task 1.0 in stage 1.0 (TID 3). 2211 bytes result sent to driver 20/08/03 12:50:38.218 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 2) in 62 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:38.219 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 1.0 (TID 3) in 61 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:38.219 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool 20/08/03 12:50:38.220 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 1 (treeAggregate at RDDLossFunction.scala:61) finished in 0.082 s 20/08/03 12:50:38.220 dag-scheduler-event-loop INFO DAGScheduler: Job 1 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:38.221 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 1: Stage finished 20/08/03 12:50:38.221 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 1 finished: treeAggregate at RDDLossFunction.scala:61, took 0.092529 s 20/08/03 12:50:38.229 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeSystemBLAS 20/08/03 12:50:38.230 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeRefBLAS 20/08/03 12:50:38.233 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(2) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:38.253 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_2_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 153.0 B, free: 2.1 GiB) 20/08/03 12:50:38.278 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_4 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:38.281 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:38.283 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_4_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:38.284 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 4 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:38.307 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:38.308 dag-scheduler-event-loop INFO DAGScheduler: Got job 2 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:38.308 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 2 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:38.308 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:38.309 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:38.310 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 2 (MapPartitionsRDD[17] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:38.314 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_5 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:38.319 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_5_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:38.320 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:38.321 dag-scheduler-event-loop INFO SparkContext: Created broadcast 5 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:38.322 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 2 (MapPartitionsRDD[17] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:38.322 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 2.0 with 2 tasks 20/08/03 12:50:38.323 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 2.0 (TID 4, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:38.324 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 2.0 (TID 5, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:38.324 Executor task launch worker for task 5 INFO Executor: Running task 1.0 in stage 2.0 (TID 5) 20/08/03 12:50:38.324 Executor task launch worker for task 4 INFO Executor: Running task 0.0 in stage 2.0 (TID 4) 20/08/03 12:50:38.330 Executor task launch worker for task 4 INFO BlockManager: Found block rdd_9_0 locally 20/08/03 12:50:38.330 Executor task launch worker for task 5 INFO BlockManager: Found block rdd_9_1 locally 20/08/03 12:50:38.333 Executor task launch worker for task 4 INFO Executor: Finished task 0.0 in stage 2.0 (TID 4). 2211 bytes result sent to driver 20/08/03 12:50:38.334 Executor task launch worker for task 5 INFO Executor: Finished task 1.0 in stage 2.0 (TID 5). 2211 bytes result sent to driver 20/08/03 12:50:38.335 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 2.0 (TID 4) in 11 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:38.336 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 2.0 (TID 5) in 13 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:38.336 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 2.0, whose tasks have all completed, from pool 20/08/03 12:50:38.336 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 2 (treeAggregate at RDDLossFunction.scala:61) finished in 0.025 s 20/08/03 12:50:38.337 dag-scheduler-event-loop INFO DAGScheduler: Job 2 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:38.337 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 2: Stage finished 20/08/03 12:50:38.337 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 2 finished: treeAggregate at RDDLossFunction.scala:61, took 0.030210 s 20/08/03 12:50:38.338 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(4) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:38.340 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_4_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:38.341 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 3.941 20/08/03 12:50:38.342 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.493391 (rel: 0.230) 0.0621158 20/08/03 12:50:38.345 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_6 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:38.348 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_6_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:38.348 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:38.349 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 6 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:38.370 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:38.372 dag-scheduler-event-loop INFO DAGScheduler: Got job 3 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:38.372 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 3 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:38.372 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:38.373 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:38.374 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 3 (MapPartitionsRDD[18] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:38.377 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_7 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:38.381 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_7_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:38.382 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_7_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:38.382 dag-scheduler-event-loop INFO SparkContext: Created broadcast 7 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:38.383 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 3 (MapPartitionsRDD[18] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:38.383 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 3.0 with 2 tasks 20/08/03 12:50:38.385 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 3.0 (TID 6, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:38.386 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 3.0 (TID 7, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:38.386 Executor task launch worker for task 7 INFO Executor: Running task 1.0 in stage 3.0 (TID 7) 20/08/03 12:50:38.386 Executor task launch worker for task 6 INFO Executor: Running task 0.0 in stage 3.0 (TID 6) 20/08/03 12:50:38.392 Executor task launch worker for task 7 INFO BlockManager: Found block rdd_9_1 locally 20/08/03 12:50:38.392 Executor task launch worker for task 6 INFO BlockManager: Found block rdd_9_0 locally 20/08/03 12:50:38.394 Executor task launch worker for task 7 INFO Executor: Finished task 1.0 in stage 3.0 (TID 7). 2211 bytes result sent to driver 20/08/03 12:50:38.395 Executor task launch worker for task 6 INFO Executor: Finished task 0.0 in stage 3.0 (TID 6). 2211 bytes result sent to driver 20/08/03 12:50:38.396 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 3.0 (TID 7) in 11 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:38.396 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 3.0 (TID 6) in 12 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:38.396 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 3.0, whose tasks have all completed, from pool 20/08/03 12:50:38.399 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 3 (treeAggregate at RDDLossFunction.scala:61) finished in 0.024 s 20/08/03 12:50:38.400 dag-scheduler-event-loop INFO DAGScheduler: Job 3 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:38.400 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 3: Stage finished 20/08/03 12:50:38.401 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 3 finished: treeAggregate at RDDLossFunction.scala:61, took 0.029855 s 20/08/03 12:50:38.402 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(6) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:38.403 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:38.403 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.478652 (rel: 0.0299) 0.0230034 20/08/03 12:50:38.404 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_6_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:38.405 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_8 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:38.408 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_8_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:38.409 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_8_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:38.409 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 8 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:38.433 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:38.435 dag-scheduler-event-loop INFO DAGScheduler: Got job 4 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:38.435 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 4 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:38.435 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:38.436 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:38.437 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 4 (MapPartitionsRDD[19] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:38.440 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_9 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:38.445 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_9_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:38.445 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:38.446 dag-scheduler-event-loop INFO SparkContext: Created broadcast 9 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:38.447 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 4 (MapPartitionsRDD[19] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:38.447 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 4.0 with 2 tasks 20/08/03 12:50:38.449 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 4.0 (TID 8, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:38.449 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 4.0 (TID 9, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:38.450 Executor task launch worker for task 8 INFO Executor: Running task 0.0 in stage 4.0 (TID 8) 20/08/03 12:50:38.450 Executor task launch worker for task 9 INFO Executor: Running task 1.0 in stage 4.0 (TID 9) 20/08/03 12:50:38.457 Executor task launch worker for task 9 INFO BlockManager: Found block rdd_9_1 locally 20/08/03 12:50:38.457 Executor task launch worker for task 8 INFO BlockManager: Found block rdd_9_0 locally 20/08/03 12:50:38.459 Executor task launch worker for task 9 INFO Executor: Finished task 1.0 in stage 4.0 (TID 9). 2211 bytes result sent to driver 20/08/03 12:50:38.460 Executor task launch worker for task 8 INFO Executor: Finished task 0.0 in stage 4.0 (TID 8). 2211 bytes result sent to driver 20/08/03 12:50:38.461 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 4.0 (TID 9) in 12 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:38.461 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 4.0 (TID 8) in 13 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:38.461 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 4.0, whose tasks have all completed, from pool 20/08/03 12:50:38.462 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 4 (treeAggregate at RDDLossFunction.scala:61) finished in 0.024 s 20/08/03 12:50:38.463 dag-scheduler-event-loop INFO DAGScheduler: Job 4 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:38.463 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 4: Stage finished 20/08/03 12:50:38.463 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 4 finished: treeAggregate at RDDLossFunction.scala:61, took 0.030233 s 20/08/03 12:50:38.464 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(8) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:38.466 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:38.466 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.475811 (rel: 0.00594) 0.00507866 20/08/03 12:50:38.467 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_8_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:38.468 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_10 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:38.470 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_10_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:38.471 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:38.472 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 10 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:38.491 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:38.492 dag-scheduler-event-loop INFO DAGScheduler: Got job 5 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:38.492 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 5 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:38.493 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:38.493 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:38.494 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 5 (MapPartitionsRDD[20] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:38.497 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_11 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:38.501 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_11_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:38.501 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:38.502 dag-scheduler-event-loop INFO SparkContext: Created broadcast 11 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:38.503 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 5 (MapPartitionsRDD[20] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:38.503 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 5.0 with 2 tasks 20/08/03 12:50:38.505 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 5.0 (TID 10, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:38.505 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 5.0 (TID 11, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:38.506 Executor task launch worker for task 11 INFO Executor: Running task 1.0 in stage 5.0 (TID 11) 20/08/03 12:50:38.506 Executor task launch worker for task 10 INFO Executor: Running task 0.0 in stage 5.0 (TID 10) 20/08/03 12:50:38.512 Executor task launch worker for task 10 INFO BlockManager: Found block rdd_9_0 locally 20/08/03 12:50:38.512 Executor task launch worker for task 11 INFO BlockManager: Found block rdd_9_1 locally 20/08/03 12:50:38.515 Executor task launch worker for task 10 INFO Executor: Finished task 0.0 in stage 5.0 (TID 10). 2211 bytes result sent to driver 20/08/03 12:50:38.515 Executor task launch worker for task 11 INFO Executor: Finished task 1.0 in stage 5.0 (TID 11). 2211 bytes result sent to driver 20/08/03 12:50:38.517 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 5.0 (TID 10) in 13 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:38.517 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 5.0 (TID 11) in 12 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:38.517 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 5.0, whose tasks have all completed, from pool 20/08/03 12:50:38.518 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 5 (treeAggregate at RDDLossFunction.scala:61) finished in 0.023 s 20/08/03 12:50:38.518 dag-scheduler-event-loop INFO DAGScheduler: Job 5 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:38.519 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 5: Stage finished 20/08/03 12:50:38.519 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 5 finished: treeAggregate at RDDLossFunction.scala:61, took 0.027446 s 20/08/03 12:50:38.520 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(10) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:38.521 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:38.521 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.475650 (rel: 0.000337) 0.00106784 20/08/03 12:50:38.522 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_10_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:38.523 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_12 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:38.526 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_12_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:38.526 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:38.527 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 12 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:38.548 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:38.549 dag-scheduler-event-loop INFO DAGScheduler: Got job 6 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:38.549 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 6 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:38.549 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:38.550 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:38.551 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 6 (MapPartitionsRDD[21] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:38.555 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_13 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:38.559 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_13_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:38.560 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:38.560 dag-scheduler-event-loop INFO SparkContext: Created broadcast 13 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:38.561 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 6 (MapPartitionsRDD[21] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:38.561 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 6.0 with 2 tasks 20/08/03 12:50:38.563 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 6.0 (TID 12, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:38.563 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 6.0 (TID 13, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:38.564 Executor task launch worker for task 12 INFO Executor: Running task 0.0 in stage 6.0 (TID 12) 20/08/03 12:50:38.564 Executor task launch worker for task 13 INFO Executor: Running task 1.0 in stage 6.0 (TID 13) 20/08/03 12:50:38.568 Executor task launch worker for task 12 INFO BlockManager: Found block rdd_9_0 locally 20/08/03 12:50:38.568 Executor task launch worker for task 13 INFO BlockManager: Found block rdd_9_1 locally 20/08/03 12:50:38.570 Executor task launch worker for task 13 INFO Executor: Finished task 1.0 in stage 6.0 (TID 13). 2211 bytes result sent to driver 20/08/03 12:50:38.570 Executor task launch worker for task 12 INFO Executor: Finished task 0.0 in stage 6.0 (TID 12). 2211 bytes result sent to driver 20/08/03 12:50:38.572 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 6.0 (TID 13) in 9 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:38.572 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 6.0 (TID 12) in 9 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:38.572 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 6.0, whose tasks have all completed, from pool 20/08/03 12:50:38.573 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 6 (treeAggregate at RDDLossFunction.scala:61) finished in 0.021 s 20/08/03 12:50:38.574 dag-scheduler-event-loop INFO DAGScheduler: Job 6 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:38.574 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 6: Stage finished 20/08/03 12:50:38.574 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 6 finished: treeAggregate at RDDLossFunction.scala:61, took 0.026081 s 20/08/03 12:50:38.575 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(12) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:38.576 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:38.577 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.475647 (rel: 6.99e-06) 0.000474829 20/08/03 12:50:38.577 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_12_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:38.578 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_14 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:38.581 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_14_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:38.582 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_14_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:38.583 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 14 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:38.603 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:38.605 dag-scheduler-event-loop INFO DAGScheduler: Got job 7 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:38.605 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 7 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:38.605 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:38.606 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:38.607 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 7 (MapPartitionsRDD[22] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:38.610 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_15 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:38.615 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_15_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:38.616 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:38.616 dag-scheduler-event-loop INFO SparkContext: Created broadcast 15 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:38.617 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 7 (MapPartitionsRDD[22] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:38.617 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 7.0 with 2 tasks 20/08/03 12:50:38.619 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 7.0 (TID 14, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:38.620 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 7.0 (TID 15, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:38.621 Executor task launch worker for task 15 INFO Executor: Running task 1.0 in stage 7.0 (TID 15) 20/08/03 12:50:38.621 Executor task launch worker for task 14 INFO Executor: Running task 0.0 in stage 7.0 (TID 14) 20/08/03 12:50:38.625 Executor task launch worker for task 15 INFO BlockManager: Found block rdd_9_1 locally 20/08/03 12:50:38.627 Executor task launch worker for task 15 INFO Executor: Finished task 1.0 in stage 7.0 (TID 15). 2211 bytes result sent to driver 20/08/03 12:50:38.628 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 7.0 (TID 15) in 9 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:38.634 Executor task launch worker for task 14 INFO BlockManager: Found block rdd_9_0 locally 20/08/03 12:50:38.636 Executor task launch worker for task 14 INFO Executor: Finished task 0.0 in stage 7.0 (TID 14). 2211 bytes result sent to driver 20/08/03 12:50:38.638 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 7.0 (TID 14) in 19 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:38.638 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 7.0, whose tasks have all completed, from pool 20/08/03 12:50:38.639 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 7 (treeAggregate at RDDLossFunction.scala:61) finished in 0.031 s 20/08/03 12:50:38.639 dag-scheduler-event-loop INFO DAGScheduler: Job 7 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:38.639 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 7: Stage finished 20/08/03 12:50:38.640 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 7 finished: treeAggregate at RDDLossFunction.scala:61, took 0.035911 s 20/08/03 12:50:38.640 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(14) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:38.641 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:38.642 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.475647 (rel: 1.30e-06) 3.45553e-06 20/08/03 12:50:38.642 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_14_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:38.643 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_16 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:38.646 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_16_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:38.646 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_16_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:38.647 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 16 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:38.667 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:38.668 dag-scheduler-event-loop INFO DAGScheduler: Got job 8 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:38.668 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 8 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:38.668 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:38.669 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:38.670 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 8 (MapPartitionsRDD[23] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:38.673 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_17 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:38.677 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_17_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:38.677 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_17_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:38.678 dag-scheduler-event-loop INFO SparkContext: Created broadcast 17 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:38.679 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 8 (MapPartitionsRDD[23] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:38.679 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 8.0 with 2 tasks 20/08/03 12:50:38.681 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 8.0 (TID 16, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:38.682 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 8.0 (TID 17, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:38.682 Executor task launch worker for task 17 INFO Executor: Running task 1.0 in stage 8.0 (TID 17) 20/08/03 12:50:38.682 Executor task launch worker for task 16 INFO Executor: Running task 0.0 in stage 8.0 (TID 16) 20/08/03 12:50:38.688 Executor task launch worker for task 17 INFO BlockManager: Found block rdd_9_1 locally 20/08/03 12:50:38.688 Executor task launch worker for task 16 INFO BlockManager: Found block rdd_9_0 locally 20/08/03 12:50:38.690 Executor task launch worker for task 17 INFO Executor: Finished task 1.0 in stage 8.0 (TID 17). 2211 bytes result sent to driver 20/08/03 12:50:38.690 Executor task launch worker for task 16 INFO Executor: Finished task 0.0 in stage 8.0 (TID 16). 2211 bytes result sent to driver 20/08/03 12:50:38.692 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 8.0 (TID 17) in 10 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:38.692 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 8.0 (TID 16) in 11 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:38.692 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 8.0, whose tasks have all completed, from pool 20/08/03 12:50:38.693 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 8 (treeAggregate at RDDLossFunction.scala:61) finished in 0.022 s 20/08/03 12:50:38.693 dag-scheduler-event-loop INFO DAGScheduler: Job 8 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:38.693 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 8: Stage finished 20/08/03 12:50:38.693 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 8 finished: treeAggregate at RDDLossFunction.scala:61, took 0.026140 s 20/08/03 12:50:38.694 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(16) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:38.695 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:38.695 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.475647 (rel: 9.23e-11) 6.96515e-08 20/08/03 12:50:38.696 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_16_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:38.698 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Converged because gradient converged 20/08/03 12:50:38.699 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(1) (from destroy at LogisticRegression.scala:804) 20/08/03 12:50:38.700 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_1_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 77.0 B, free: 2.1 GiB) 20/08/03 12:50:38.702 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MapPartitionsRDD: Removing RDD 9 from persistence list 20/08/03 12:50:38.708 block-manager-slave-async-thread-pool-27 INFO BlockManager: Removing RDD 9 20/08/03 12:50:38.866 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 9.630614 ms 20/08/03 12:50:38.962 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 19.085896 ms 20/08/03 12:50:38.983 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [ccf2a6df] training finished 20/08/03 12:50:39.104 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 10.169591 ms 20/08/03 12:50:39.117 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 8.84745 ms 20/08/03 12:50:39.187 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 8.204887 ms 20/08/03 12:50:39.197 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 6.9871 ms 20/08/03 12:50:39.320 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 22.813142 ms 20/08/03 12:50:39.339 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 12.387332 ms 20/08/03 12:50:39.349 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: ===== FINISHED o.a.s.ml.classification.LogisticRegressionSuite: 'logistic regression: default params' ===== 20/08/03 12:50:39.350 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: ===== TEST OUTPUT FOR o.a.s.ml.classification.LogisticRegressionSuite: 'logistic regression: illegal params' ===== 20/08/03 12:50:39.480 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 28.962566 ms 20/08/03 12:50:39.494 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [24c64493] Stage class: LogisticRegression 20/08/03 12:50:39.494 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [24c64493] Stage uid: logreg_9c38c03a8f8b 20/08/03 12:50:39.562 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 24.554893 ms 20/08/03 12:50:39.571 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [24c64493] training: numPartitions=4 storageLevel=StorageLevel(1 replicas) 20/08/03 12:50:39.572 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [24c64493] {} 20/08/03 12:50:39.583 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at LogisticRegression.scala:499 20/08/03 12:50:39.585 dag-scheduler-event-loop INFO DAGScheduler: Got job 9 (treeAggregate at LogisticRegression.scala:499) with 4 output partitions 20/08/03 12:50:39.585 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 9 (treeAggregate at LogisticRegression.scala:499) 20/08/03 12:50:39.585 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:39.586 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:39.587 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 9 (MapPartitionsRDD[48] at treeAggregate at LogisticRegression.scala:499), which has no missing parents 20/08/03 12:50:39.613 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_18 stored as values in memory (estimated size 27.5 KiB, free 2.1 GiB) 20/08/03 12:50:39.619 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_18_piece0 stored as bytes in memory (estimated size 10.7 KiB, free 2.1 GiB) 20/08/03 12:50:39.620 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_18_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.7 KiB, free: 2.1 GiB) 20/08/03 12:50:39.621 dag-scheduler-event-loop INFO SparkContext: Created broadcast 18 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:39.621 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 9 (MapPartitionsRDD[48] at treeAggregate at LogisticRegression.scala:499) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:39.621 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 9.0 with 4 tasks 20/08/03 12:50:39.628 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 9.0 (TID 18, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:39.635 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 9.0 (TID 19, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:39.636 Executor task launch worker for task 18 INFO Executor: Running task 0.0 in stage 9.0 (TID 18) 20/08/03 12:50:39.636 Executor task launch worker for task 19 INFO Executor: Running task 1.0 in stage 9.0 (TID 19) 20/08/03 12:50:39.900 Executor task launch worker for task 18 INFO MemoryStore: Block rdd_42_0 stored as values in memory (estimated size 244.2 KiB, free 2.1 GiB) 20/08/03 12:50:39.901 Executor task launch worker for task 19 INFO MemoryStore: Block rdd_42_1 stored as values in memory (estimated size 244.2 KiB, free 2.1 GiB) 20/08/03 12:50:39.901 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_42_0 in memory on amp-jenkins-worker-06.amp:34426 (size: 244.2 KiB, free: 2.1 GiB) 20/08/03 12:50:39.901 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_42_1 in memory on amp-jenkins-worker-06.amp:34426 (size: 244.2 KiB, free: 2.1 GiB) 20/08/03 12:50:39.928 Executor task launch worker for task 19 INFO Executor: Finished task 1.0 in stage 9.0 (TID 19). 2687 bytes result sent to driver 20/08/03 12:50:39.930 Executor task launch worker for task 18 INFO Executor: Finished task 0.0 in stage 9.0 (TID 18). 2687 bytes result sent to driver 20/08/03 12:50:39.934 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 9.0 (TID 20, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:39.935 Executor task launch worker for task 20 INFO Executor: Running task 2.0 in stage 9.0 (TID 20) 20/08/03 12:50:39.938 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 9.0 (TID 21, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:39.939 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 9.0 (TID 19) in 310 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:39.939 Executor task launch worker for task 21 INFO Executor: Running task 3.0 in stage 9.0 (TID 21) 20/08/03 12:50:39.939 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 9.0 (TID 18) in 316 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:40.012 Executor task launch worker for task 21 INFO MemoryStore: Block rdd_42_3 stored as values in memory (estimated size 244.2 KiB, free 2.1 GiB) 20/08/03 12:50:40.012 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_42_3 in memory on amp-jenkins-worker-06.amp:34426 (size: 244.2 KiB, free: 2.1 GiB) 20/08/03 12:50:40.013 Executor task launch worker for task 20 INFO MemoryStore: Block rdd_42_2 stored as values in memory (estimated size 244.2 KiB, free 2.1 GiB) 20/08/03 12:50:40.014 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_42_2 in memory on amp-jenkins-worker-06.amp:34426 (size: 244.2 KiB, free: 2.1 GiB) 20/08/03 12:50:40.021 Executor task launch worker for task 21 INFO Executor: Finished task 3.0 in stage 9.0 (TID 21). 2687 bytes result sent to driver 20/08/03 12:50:40.022 task-result-getter-0 INFO TaskSetManager: Finished task 3.0 in stage 9.0 (TID 21) in 87 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:40.025 Executor task launch worker for task 20 INFO Executor: Finished task 2.0 in stage 9.0 (TID 20). 2687 bytes result sent to driver 20/08/03 12:50:40.027 task-result-getter-1 INFO TaskSetManager: Finished task 2.0 in stage 9.0 (TID 20) in 98 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:40.027 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 9.0, whose tasks have all completed, from pool 20/08/03 12:50:40.028 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 9 (treeAggregate at LogisticRegression.scala:499) finished in 0.440 s 20/08/03 12:50:40.028 dag-scheduler-event-loop INFO DAGScheduler: Job 9 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:40.028 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 9: Stage finished 20/08/03 12:50:40.029 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 9 finished: treeAggregate at LogisticRegression.scala:499, took 0.445465 s 20/08/03 12:50:40.030 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [24c64493] {"numExamples":10000} 20/08/03 12:50:40.030 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [24c64493] {"lowestLabelWeight":"4383.0"} 20/08/03 12:50:40.030 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [24c64493] {"highestLabelWeight":"5617.0"} 20/08/03 12:50:40.031 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [24c64493] {"sumOfWeights":10000.0} 20/08/03 12:50:40.032 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [24c64493] {"numClasses":2} 20/08/03 12:50:40.032 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [24c64493] {"numFeatures":4} 20/08/03 12:50:40.033 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_19 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:40.037 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_19_piece0 stored as bytes in memory (estimated size 101.0 B, free 2.1 GiB) 20/08/03 12:50:40.037 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_19_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 101.0 B, free: 2.1 GiB) 20/08/03 12:50:40.038 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 19 from broadcast at LogisticRegression.scala:593 20/08/03 12:50:40.363 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_20 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:40.365 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_20_piece0 stored as bytes in memory (estimated size 174.0 B, free 2.1 GiB) 20/08/03 12:50:40.366 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_20_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:40.367 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 20 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:40.378 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:40.380 dag-scheduler-event-loop INFO DAGScheduler: Got job 10 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:40.380 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 10 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:40.380 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:40.381 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:40.382 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 10 (MapPartitionsRDD[49] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:40.385 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_21 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:40.390 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_21_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:40.391 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_21_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:40.391 dag-scheduler-event-loop INFO SparkContext: Created broadcast 21 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:40.392 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 10 (MapPartitionsRDD[49] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:40.392 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 10.0 with 4 tasks 20/08/03 12:50:40.396 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 10.0 (TID 22, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.398 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 10.0 (TID 23, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.398 Executor task launch worker for task 23 INFO Executor: Running task 1.0 in stage 10.0 (TID 23) 20/08/03 12:50:40.398 Executor task launch worker for task 22 INFO Executor: Running task 0.0 in stage 10.0 (TID 22) 20/08/03 12:50:40.405 Executor task launch worker for task 23 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:40.406 Executor task launch worker for task 22 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:40.419 Executor task launch worker for task 23 INFO Executor: Finished task 1.0 in stage 10.0 (TID 23). 2307 bytes result sent to driver 20/08/03 12:50:40.420 Executor task launch worker for task 22 INFO Executor: Finished task 0.0 in stage 10.0 (TID 22). 2307 bytes result sent to driver 20/08/03 12:50:40.424 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 10.0 (TID 24, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.424 Executor task launch worker for task 24 INFO Executor: Running task 2.0 in stage 10.0 (TID 24) 20/08/03 12:50:40.427 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 10.0 (TID 25, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.427 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 10.0 (TID 23) in 31 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:40.427 Executor task launch worker for task 25 INFO Executor: Running task 3.0 in stage 10.0 (TID 25) 20/08/03 12:50:40.427 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 10.0 (TID 22) in 34 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:40.438 Executor task launch worker for task 24 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:40.441 Executor task launch worker for task 24 INFO Executor: Finished task 2.0 in stage 10.0 (TID 24). 2307 bytes result sent to driver 20/08/03 12:50:40.442 Executor task launch worker for task 25 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:40.443 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 10.0 (TID 24) in 23 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:40.449 Executor task launch worker for task 25 INFO Executor: Finished task 3.0 in stage 10.0 (TID 25). 2350 bytes result sent to driver 20/08/03 12:50:40.450 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 10.0 (TID 25) in 25 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:40.451 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 10.0, whose tasks have all completed, from pool 20/08/03 12:50:40.451 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 10 (treeAggregate at RDDLossFunction.scala:61) finished in 0.068 s 20/08/03 12:50:40.452 dag-scheduler-event-loop INFO DAGScheduler: Job 10 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:40.452 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 10: Stage finished 20/08/03 12:50:40.452 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 10 finished: treeAggregate at RDDLossFunction.scala:61, took 0.073879 s 20/08/03 12:50:40.453 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(20) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:40.455 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_20_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:40.471 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_22 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:40.474 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_22_piece0 stored as bytes in memory (estimated size 174.0 B, free 2.1 GiB) 20/08/03 12:50:40.475 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_22_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:40.476 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 22 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:40.485 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:40.486 dag-scheduler-event-loop INFO DAGScheduler: Got job 11 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:40.487 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 11 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:40.487 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:40.487 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:40.488 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 11 (MapPartitionsRDD[50] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:40.491 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_23 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:40.497 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_23_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:40.497 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_23_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:40.498 dag-scheduler-event-loop INFO SparkContext: Created broadcast 23 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:40.498 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 11 (MapPartitionsRDD[50] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:40.499 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 11.0 with 4 tasks 20/08/03 12:50:40.502 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 11.0 (TID 26, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.504 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 11.0 (TID 27, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.504 Executor task launch worker for task 26 INFO Executor: Running task 0.0 in stage 11.0 (TID 26) 20/08/03 12:50:40.504 Executor task launch worker for task 27 INFO Executor: Running task 1.0 in stage 11.0 (TID 27) 20/08/03 12:50:40.510 Executor task launch worker for task 26 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:40.511 Executor task launch worker for task 27 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:40.515 Executor task launch worker for task 26 INFO Executor: Finished task 0.0 in stage 11.0 (TID 26). 2307 bytes result sent to driver 20/08/03 12:50:40.515 Executor task launch worker for task 27 INFO Executor: Finished task 1.0 in stage 11.0 (TID 27). 2307 bytes result sent to driver 20/08/03 12:50:40.519 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 11.0 (TID 28, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.520 Executor task launch worker for task 28 INFO Executor: Running task 2.0 in stage 11.0 (TID 28) 20/08/03 12:50:40.524 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 11.0 (TID 29, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.524 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 11.0 (TID 26) in 24 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:40.524 Executor task launch worker for task 29 INFO Executor: Running task 3.0 in stage 11.0 (TID 29) 20/08/03 12:50:40.524 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 11.0 (TID 27) in 22 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:40.525 Executor task launch worker for task 28 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:40.528 Executor task launch worker for task 28 INFO Executor: Finished task 2.0 in stage 11.0 (TID 28). 2307 bytes result sent to driver 20/08/03 12:50:40.529 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 11.0 (TID 28) in 13 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:40.529 Executor task launch worker for task 29 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:40.532 Executor task launch worker for task 29 INFO Executor: Finished task 3.0 in stage 11.0 (TID 29). 2307 bytes result sent to driver 20/08/03 12:50:40.532 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 11.0 (TID 29) in 12 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:40.533 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 11.0, whose tasks have all completed, from pool 20/08/03 12:50:40.533 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 11 (treeAggregate at RDDLossFunction.scala:61) finished in 0.044 s 20/08/03 12:50:40.533 dag-scheduler-event-loop INFO DAGScheduler: Job 11 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:40.533 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 11: Stage finished 20/08/03 12:50:40.534 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 11 finished: treeAggregate at RDDLossFunction.scala:61, took 0.047907 s 20/08/03 12:50:40.534 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(22) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:40.537 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_22_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:40.537 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_24 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:40.540 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_24_piece0 stored as bytes in memory (estimated size 174.0 B, free 2.1 GiB) 20/08/03 12:50:40.540 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_24_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:40.541 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 24 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:40.550 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:40.551 dag-scheduler-event-loop INFO DAGScheduler: Got job 12 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:40.551 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 12 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:40.551 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:40.552 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:40.552 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 12 (MapPartitionsRDD[51] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:40.555 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_25 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:40.561 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_25_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:40.561 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_25_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:40.562 dag-scheduler-event-loop INFO SparkContext: Created broadcast 25 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:40.562 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 12 (MapPartitionsRDD[51] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:40.562 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 12.0 with 4 tasks 20/08/03 12:50:40.565 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 12.0 (TID 30, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.567 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 12.0 (TID 31, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.568 Executor task launch worker for task 30 INFO Executor: Running task 0.0 in stage 12.0 (TID 30) 20/08/03 12:50:40.568 Executor task launch worker for task 31 INFO Executor: Running task 1.0 in stage 12.0 (TID 31) 20/08/03 12:50:40.573 Executor task launch worker for task 30 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:40.574 Executor task launch worker for task 31 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:40.575 Executor task launch worker for task 30 INFO Executor: Finished task 0.0 in stage 12.0 (TID 30). 2307 bytes result sent to driver 20/08/03 12:50:40.577 Executor task launch worker for task 31 INFO Executor: Finished task 1.0 in stage 12.0 (TID 31). 2307 bytes result sent to driver 20/08/03 12:50:40.578 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 12.0 (TID 32, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.579 Executor task launch worker for task 32 INFO Executor: Running task 2.0 in stage 12.0 (TID 32) 20/08/03 12:50:40.579 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 12.0 (TID 30) in 16 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:40.579 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 12.0 (TID 31) in 14 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:40.583 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 12.0 (TID 33, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.583 Executor task launch worker for task 33 INFO Executor: Running task 3.0 in stage 12.0 (TID 33) 20/08/03 12:50:40.588 Executor task launch worker for task 32 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:40.588 Executor task launch worker for task 33 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:40.591 Executor task launch worker for task 33 INFO Executor: Finished task 3.0 in stage 12.0 (TID 33). 2307 bytes result sent to driver 20/08/03 12:50:40.592 Executor task launch worker for task 32 INFO Executor: Finished task 2.0 in stage 12.0 (TID 32). 2307 bytes result sent to driver 20/08/03 12:50:40.593 task-result-getter-0 INFO TaskSetManager: Finished task 3.0 in stage 12.0 (TID 33) in 13 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:40.593 task-result-getter-1 INFO TaskSetManager: Finished task 2.0 in stage 12.0 (TID 32) in 17 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:40.593 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 12.0, whose tasks have all completed, from pool 20/08/03 12:50:40.593 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 12 (treeAggregate at RDDLossFunction.scala:61) finished in 0.040 s 20/08/03 12:50:40.594 dag-scheduler-event-loop INFO DAGScheduler: Job 12 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:40.594 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 12: Stage finished 20/08/03 12:50:40.594 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 12 finished: treeAggregate at RDDLossFunction.scala:61, took 0.043720 s 20/08/03 12:50:40.595 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(24) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:40.597 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_26 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:40.597 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_24_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:40.599 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_26_piece0 stored as bytes in memory (estimated size 174.0 B, free 2.1 GiB) 20/08/03 12:50:40.599 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_26_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:40.600 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 26 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:40.609 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:40.610 dag-scheduler-event-loop INFO DAGScheduler: Got job 13 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:40.610 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 13 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:40.610 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:40.611 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:40.611 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 13 (MapPartitionsRDD[52] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:40.615 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_27 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:40.620 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_27_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:40.621 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_27_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:40.621 dag-scheduler-event-loop INFO SparkContext: Created broadcast 27 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:40.622 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 13 (MapPartitionsRDD[52] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:40.622 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 13.0 with 4 tasks 20/08/03 12:50:40.625 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 13.0 (TID 34, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.627 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 13.0 (TID 35, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.627 Executor task launch worker for task 35 INFO Executor: Running task 1.0 in stage 13.0 (TID 35) 20/08/03 12:50:40.627 Executor task launch worker for task 34 INFO Executor: Running task 0.0 in stage 13.0 (TID 34) 20/08/03 12:50:40.632 Executor task launch worker for task 35 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:40.635 Executor task launch worker for task 34 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:40.635 Executor task launch worker for task 35 INFO Executor: Finished task 1.0 in stage 13.0 (TID 35). 2307 bytes result sent to driver 20/08/03 12:50:40.638 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 13.0 (TID 36, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.638 Executor task launch worker for task 36 INFO Executor: Running task 2.0 in stage 13.0 (TID 36) 20/08/03 12:50:40.639 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 13.0 (TID 35) in 14 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:40.639 Executor task launch worker for task 34 INFO Executor: Finished task 0.0 in stage 13.0 (TID 34). 2307 bytes result sent to driver 20/08/03 12:50:40.642 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 13.0 (TID 37, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.643 Executor task launch worker for task 37 INFO Executor: Running task 3.0 in stage 13.0 (TID 37) 20/08/03 12:50:40.643 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 13.0 (TID 34) in 20 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:40.645 Executor task launch worker for task 36 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:40.648 Executor task launch worker for task 36 INFO Executor: Finished task 2.0 in stage 13.0 (TID 36). 2307 bytes result sent to driver 20/08/03 12:50:40.649 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 13.0 (TID 36) in 13 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:40.650 Executor task launch worker for task 37 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:40.653 Executor task launch worker for task 37 INFO Executor: Finished task 3.0 in stage 13.0 (TID 37). 2307 bytes result sent to driver 20/08/03 12:50:40.654 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 13.0 (TID 37) in 15 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:40.654 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 13.0, whose tasks have all completed, from pool 20/08/03 12:50:40.655 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 13 (treeAggregate at RDDLossFunction.scala:61) finished in 0.041 s 20/08/03 12:50:40.655 dag-scheduler-event-loop INFO DAGScheduler: Job 13 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:40.655 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 13: Stage finished 20/08/03 12:50:40.655 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 13 finished: treeAggregate at RDDLossFunction.scala:61, took 0.045868 s 20/08/03 12:50:40.656 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(26) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:40.658 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_28 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:40.658 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_26_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:40.660 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_28_piece0 stored as bytes in memory (estimated size 174.0 B, free 2.1 GiB) 20/08/03 12:50:40.660 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_28_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:40.661 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 28 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:40.670 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:40.671 dag-scheduler-event-loop INFO DAGScheduler: Got job 14 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:40.671 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 14 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:40.671 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:40.672 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:40.672 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 14 (MapPartitionsRDD[53] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:40.675 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_29 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:40.681 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_29_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:40.682 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_29_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:40.682 dag-scheduler-event-loop INFO SparkContext: Created broadcast 29 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:40.683 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 14 (MapPartitionsRDD[53] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:40.683 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 14.0 with 4 tasks 20/08/03 12:50:40.686 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 14.0 (TID 38, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.688 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 14.0 (TID 39, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.688 Executor task launch worker for task 38 INFO Executor: Running task 0.0 in stage 14.0 (TID 38) 20/08/03 12:50:40.689 Executor task launch worker for task 39 INFO Executor: Running task 1.0 in stage 14.0 (TID 39) 20/08/03 12:50:40.693 Executor task launch worker for task 39 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:40.696 Executor task launch worker for task 38 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:40.696 Executor task launch worker for task 39 INFO Executor: Finished task 1.0 in stage 14.0 (TID 39). 2307 bytes result sent to driver 20/08/03 12:50:40.699 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 14.0 (TID 40, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.699 Executor task launch worker for task 38 INFO Executor: Finished task 0.0 in stage 14.0 (TID 38). 2307 bytes result sent to driver 20/08/03 12:50:40.700 Executor task launch worker for task 40 INFO Executor: Running task 2.0 in stage 14.0 (TID 40) 20/08/03 12:50:40.704 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 14.0 (TID 41, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.705 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 14.0 (TID 39) in 19 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:40.705 Executor task launch worker for task 41 INFO Executor: Running task 3.0 in stage 14.0 (TID 41) 20/08/03 12:50:40.705 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 14.0 (TID 38) in 21 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:40.707 Executor task launch worker for task 40 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:40.709 Executor task launch worker for task 40 INFO Executor: Finished task 2.0 in stage 14.0 (TID 40). 2307 bytes result sent to driver 20/08/03 12:50:40.710 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 14.0 (TID 40) in 13 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:40.712 Executor task launch worker for task 41 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:40.716 Executor task launch worker for task 41 INFO Executor: Finished task 3.0 in stage 14.0 (TID 41). 2307 bytes result sent to driver 20/08/03 12:50:40.717 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 14.0 (TID 41) in 16 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:40.717 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 14.0, whose tasks have all completed, from pool 20/08/03 12:50:40.717 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 14 (treeAggregate at RDDLossFunction.scala:61) finished in 0.044 s 20/08/03 12:50:40.717 dag-scheduler-event-loop INFO DAGScheduler: Job 14 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:40.717 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 14: Stage finished 20/08/03 12:50:40.718 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 14 finished: treeAggregate at RDDLossFunction.scala:61, took 0.047593 s 20/08/03 12:50:40.718 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(28) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:40.720 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_30 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:40.720 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_28_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:40.724 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_30_piece0 stored as bytes in memory (estimated size 174.0 B, free 2.1 GiB) 20/08/03 12:50:40.724 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_30_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:40.725 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 30 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:40.735 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:40.736 dag-scheduler-event-loop INFO DAGScheduler: Got job 15 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:40.736 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 15 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:40.736 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:40.737 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:40.737 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 15 (MapPartitionsRDD[54] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:40.741 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_31 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:40.749 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_31_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:40.749 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_31_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:40.750 dag-scheduler-event-loop INFO SparkContext: Created broadcast 31 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:40.752 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 15 (MapPartitionsRDD[54] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:40.752 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 15.0 with 4 tasks 20/08/03 12:50:40.755 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 15.0 (TID 42, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.757 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 15.0 (TID 43, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.757 Executor task launch worker for task 43 INFO Executor: Running task 1.0 in stage 15.0 (TID 43) 20/08/03 12:50:40.757 Executor task launch worker for task 42 INFO Executor: Running task 0.0 in stage 15.0 (TID 42) 20/08/03 12:50:40.765 Executor task launch worker for task 43 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:40.765 Executor task launch worker for task 42 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:40.768 Executor task launch worker for task 42 INFO Executor: Finished task 0.0 in stage 15.0 (TID 42). 2307 bytes result sent to driver 20/08/03 12:50:40.768 Executor task launch worker for task 43 INFO Executor: Finished task 1.0 in stage 15.0 (TID 43). 2307 bytes result sent to driver 20/08/03 12:50:40.772 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 15.0 (TID 44, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.773 Executor task launch worker for task 44 INFO Executor: Running task 2.0 in stage 15.0 (TID 44) 20/08/03 12:50:40.776 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 15.0 (TID 45, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.777 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 15.0 (TID 42) in 24 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:40.777 Executor task launch worker for task 45 INFO Executor: Running task 3.0 in stage 15.0 (TID 45) 20/08/03 12:50:40.777 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 15.0 (TID 43) in 22 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:40.780 Executor task launch worker for task 44 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:40.782 Executor task launch worker for task 45 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:40.784 Executor task launch worker for task 44 INFO Executor: Finished task 2.0 in stage 15.0 (TID 44). 2307 bytes result sent to driver 20/08/03 12:50:40.786 Executor task launch worker for task 45 INFO Executor: Finished task 3.0 in stage 15.0 (TID 45). 2350 bytes result sent to driver 20/08/03 12:50:40.786 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 15.0 (TID 44) in 17 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:40.787 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 15.0 (TID 45) in 13 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:40.787 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 15.0, whose tasks have all completed, from pool 20/08/03 12:50:40.788 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 15 (treeAggregate at RDDLossFunction.scala:61) finished in 0.048 s 20/08/03 12:50:40.788 dag-scheduler-event-loop INFO DAGScheduler: Job 15 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:40.788 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 15: Stage finished 20/08/03 12:50:40.788 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 15 finished: treeAggregate at RDDLossFunction.scala:61, took 0.052684 s 20/08/03 12:50:40.789 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(30) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:40.791 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_30_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:40.791 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_32 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:40.809 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_32_piece0 stored as bytes in memory (estimated size 174.0 B, free 2.1 GiB) 20/08/03 12:50:40.810 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_32_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:40.811 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 32 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:40.818 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_9_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:40.821 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_25_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:40.823 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:40.824 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_31_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:40.825 dag-scheduler-event-loop INFO DAGScheduler: Got job 16 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:40.825 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 16 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:40.825 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:40.825 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:40.826 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 16 (MapPartitionsRDD[55] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:40.827 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_7_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:40.828 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_33 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:40.831 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_23_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:40.832 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_33_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:40.832 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_33_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:40.833 dag-scheduler-event-loop INFO SparkContext: Created broadcast 33 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:40.833 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 16 (MapPartitionsRDD[55] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:40.833 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 16.0 with 4 tasks 20/08/03 12:50:40.836 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 16.0 (TID 46, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.838 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 16.0 (TID 47, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.838 Executor task launch worker for task 46 INFO Executor: Running task 0.0 in stage 16.0 (TID 46) 20/08/03 12:50:40.838 Executor task launch worker for task 47 INFO Executor: Running task 1.0 in stage 16.0 (TID 47) 20/08/03 12:50:40.841 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_29_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:40.844 Executor task launch worker for task 46 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:40.844 Executor task launch worker for task 47 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:40.846 Executor task launch worker for task 46 INFO Executor: Finished task 0.0 in stage 16.0 (TID 46). 2307 bytes result sent to driver 20/08/03 12:50:40.847 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_27_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:40.847 Executor task launch worker for task 47 INFO Executor: Finished task 1.0 in stage 16.0 (TID 47). 2307 bytes result sent to driver 20/08/03 12:50:40.849 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 16.0 (TID 48, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.849 Executor task launch worker for task 48 INFO Executor: Running task 2.0 in stage 16.0 (TID 48) 20/08/03 12:50:40.851 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 16.0 (TID 49, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.851 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 16.0 (TID 46) in 17 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:40.851 Executor task launch worker for task 49 INFO Executor: Running task 3.0 in stage 16.0 (TID 49) 20/08/03 12:50:40.852 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 16.0 (TID 47) in 16 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:40.854 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_11_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:40.856 Executor task launch worker for task 49 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:40.856 Executor task launch worker for task 48 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:40.858 Executor task launch worker for task 49 INFO Executor: Finished task 3.0 in stage 16.0 (TID 49). 2307 bytes result sent to driver 20/08/03 12:50:40.858 Executor task launch worker for task 48 INFO Executor: Finished task 2.0 in stage 16.0 (TID 48). 2307 bytes result sent to driver 20/08/03 12:50:40.859 task-result-getter-0 INFO TaskSetManager: Finished task 3.0 in stage 16.0 (TID 49) in 10 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:40.859 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_21_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:40.860 task-result-getter-1 INFO TaskSetManager: Finished task 2.0 in stage 16.0 (TID 48) in 14 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:40.860 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 16.0, whose tasks have all completed, from pool 20/08/03 12:50:40.861 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 16 (treeAggregate at RDDLossFunction.scala:61) finished in 0.033 s 20/08/03 12:50:40.861 dag-scheduler-event-loop INFO DAGScheduler: Job 16 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:40.861 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 16: Stage finished 20/08/03 12:50:40.861 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 16 finished: treeAggregate at RDDLossFunction.scala:61, took 0.037637 s 20/08/03 12:50:40.862 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(32) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:40.863 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_32_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:40.863 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_34 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:40.866 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_34_piece0 stored as bytes in memory (estimated size 174.0 B, free 2.1 GiB) 20/08/03 12:50:40.866 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_13_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:40.866 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_34_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:40.867 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 34 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:40.868 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_15_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:40.870 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_17_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:40.873 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_18_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.7 KiB, free: 2.1 GiB) 20/08/03 12:50:40.876 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:40.876 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_3_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:40.877 dag-scheduler-event-loop INFO DAGScheduler: Got job 17 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:40.877 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 17 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:40.877 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:40.877 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:40.878 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 17 (MapPartitionsRDD[56] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:40.879 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_5_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:40.881 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_35 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:40.883 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_35_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:40.884 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_35_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:40.884 dag-scheduler-event-loop INFO SparkContext: Created broadcast 35 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:40.884 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 17 (MapPartitionsRDD[56] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:40.884 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 17.0 with 4 tasks 20/08/03 12:50:40.887 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 17.0 (TID 50, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.889 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 17.0 (TID 51, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.890 Executor task launch worker for task 51 INFO Executor: Running task 1.0 in stage 17.0 (TID 51) 20/08/03 12:50:40.890 Executor task launch worker for task 50 INFO Executor: Running task 0.0 in stage 17.0 (TID 50) 20/08/03 12:50:40.895 Executor task launch worker for task 51 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:40.895 Executor task launch worker for task 50 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:40.897 Executor task launch worker for task 51 INFO Executor: Finished task 1.0 in stage 17.0 (TID 51). 2307 bytes result sent to driver 20/08/03 12:50:40.897 Executor task launch worker for task 50 INFO Executor: Finished task 0.0 in stage 17.0 (TID 50). 2307 bytes result sent to driver 20/08/03 12:50:40.899 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 17.0 (TID 52, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.899 Executor task launch worker for task 52 INFO Executor: Running task 2.0 in stage 17.0 (TID 52) 20/08/03 12:50:40.902 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 17.0 (TID 53, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:40.902 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 17.0 (TID 51) in 14 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:40.902 Executor task launch worker for task 53 INFO Executor: Running task 3.0 in stage 17.0 (TID 53) 20/08/03 12:50:40.902 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 17.0 (TID 50) in 17 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:40.904 Executor task launch worker for task 52 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:40.906 Executor task launch worker for task 52 INFO Executor: Finished task 2.0 in stage 17.0 (TID 52). 2307 bytes result sent to driver 20/08/03 12:50:40.907 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 17.0 (TID 52) in 10 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:40.909 Executor task launch worker for task 53 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:40.912 Executor task launch worker for task 53 INFO Executor: Finished task 3.0 in stage 17.0 (TID 53). 2307 bytes result sent to driver 20/08/03 12:50:40.913 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 17.0 (TID 53) in 14 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:40.913 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 17.0, whose tasks have all completed, from pool 20/08/03 12:50:40.914 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 17 (treeAggregate at RDDLossFunction.scala:61) finished in 0.035 s 20/08/03 12:50:40.914 dag-scheduler-event-loop INFO DAGScheduler: Job 17 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:40.914 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 17: Stage finished 20/08/03 12:50:40.915 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 17 finished: treeAggregate at RDDLossFunction.scala:61, took 0.038767 s 20/08/03 12:50:40.916 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(34) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:40.916 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Step Size: 11.39 20/08/03 12:50:40.916 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Val and Grad Norm: 2.16572 (rel: 0.612) 4.65486 20/08/03 12:50:40.918 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_34_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:40.955 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite WARN LAPACK: Failed to load implementation from: com.github.fommil.netlib.NativeSystemLAPACK 20/08/03 12:50:40.956 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite WARN LAPACK: Failed to load implementation from: com.github.fommil.netlib.NativeRefLAPACK 20/08/03 12:50:41.041 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_36 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:41.043 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_36_piece0 stored as bytes in memory (estimated size 174.0 B, free 2.1 GiB) 20/08/03 12:50:41.044 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_36_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:41.044 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 36 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:41.052 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:41.054 dag-scheduler-event-loop INFO DAGScheduler: Got job 18 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:41.054 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 18 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:41.054 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:41.054 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:41.057 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 18 (MapPartitionsRDD[57] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:41.061 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_37 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:41.064 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_37_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:41.064 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_37_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:41.065 dag-scheduler-event-loop INFO SparkContext: Created broadcast 37 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:41.065 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 18 (MapPartitionsRDD[57] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:41.065 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 18.0 with 4 tasks 20/08/03 12:50:41.069 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 18.0 (TID 54, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.072 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 18.0 (TID 55, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.072 Executor task launch worker for task 55 INFO Executor: Running task 1.0 in stage 18.0 (TID 55) 20/08/03 12:50:41.072 Executor task launch worker for task 54 INFO Executor: Running task 0.0 in stage 18.0 (TID 54) 20/08/03 12:50:41.079 Executor task launch worker for task 55 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:41.079 Executor task launch worker for task 54 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:41.081 Executor task launch worker for task 55 INFO Executor: Finished task 1.0 in stage 18.0 (TID 55). 2307 bytes result sent to driver 20/08/03 12:50:41.081 Executor task launch worker for task 54 INFO Executor: Finished task 0.0 in stage 18.0 (TID 54). 2307 bytes result sent to driver 20/08/03 12:50:41.086 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 18.0 (TID 56, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.087 Executor task launch worker for task 56 INFO Executor: Running task 2.0 in stage 18.0 (TID 56) 20/08/03 12:50:41.089 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 18.0 (TID 57, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.089 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 18.0 (TID 55) in 20 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:41.089 Executor task launch worker for task 57 INFO Executor: Running task 3.0 in stage 18.0 (TID 57) 20/08/03 12:50:41.089 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 18.0 (TID 54) in 22 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:41.092 Executor task launch worker for task 56 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:41.094 Executor task launch worker for task 57 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:41.094 Executor task launch worker for task 56 INFO Executor: Finished task 2.0 in stage 18.0 (TID 56). 2307 bytes result sent to driver 20/08/03 12:50:41.095 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 18.0 (TID 56) in 13 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:41.096 Executor task launch worker for task 57 INFO Executor: Finished task 3.0 in stage 18.0 (TID 57). 2307 bytes result sent to driver 20/08/03 12:50:41.097 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 18.0 (TID 57) in 10 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:41.097 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 18.0, whose tasks have all completed, from pool 20/08/03 12:50:41.097 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 18 (treeAggregate at RDDLossFunction.scala:61) finished in 0.039 s 20/08/03 12:50:41.098 dag-scheduler-event-loop INFO DAGScheduler: Job 18 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:41.098 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 18: Stage finished 20/08/03 12:50:41.099 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 18 finished: treeAggregate at RDDLossFunction.scala:61, took 0.045910 s 20/08/03 12:50:41.099 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(36) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:41.101 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_36_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:41.103 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_38 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:41.105 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_38_piece0 stored as bytes in memory (estimated size 174.0 B, free 2.1 GiB) 20/08/03 12:50:41.105 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_38_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:41.106 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 38 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:41.116 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:41.117 dag-scheduler-event-loop INFO DAGScheduler: Got job 19 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:41.117 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 19 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:41.117 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:41.117 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:41.118 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 19 (MapPartitionsRDD[58] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:41.121 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_39 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:41.124 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_39_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:41.124 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_39_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:41.124 dag-scheduler-event-loop INFO SparkContext: Created broadcast 39 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:41.129 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 19 (MapPartitionsRDD[58] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:41.129 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 19.0 with 4 tasks 20/08/03 12:50:41.132 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 19.0 (TID 58, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.134 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 19.0 (TID 59, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.135 Executor task launch worker for task 58 INFO Executor: Running task 0.0 in stage 19.0 (TID 58) 20/08/03 12:50:41.135 Executor task launch worker for task 59 INFO Executor: Running task 1.0 in stage 19.0 (TID 59) 20/08/03 12:50:41.140 Executor task launch worker for task 58 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:41.140 Executor task launch worker for task 59 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:41.142 Executor task launch worker for task 59 INFO Executor: Finished task 1.0 in stage 19.0 (TID 59). 2307 bytes result sent to driver 20/08/03 12:50:41.142 Executor task launch worker for task 58 INFO Executor: Finished task 0.0 in stage 19.0 (TID 58). 2307 bytes result sent to driver 20/08/03 12:50:41.145 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 19.0 (TID 60, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.145 Executor task launch worker for task 60 INFO Executor: Running task 2.0 in stage 19.0 (TID 60) 20/08/03 12:50:41.147 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 19.0 (TID 61, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.147 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 19.0 (TID 59) in 15 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:41.147 Executor task launch worker for task 61 INFO Executor: Running task 3.0 in stage 19.0 (TID 61) 20/08/03 12:50:41.148 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 19.0 (TID 58) in 18 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:41.149 Executor task launch worker for task 60 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:41.151 Executor task launch worker for task 61 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:41.151 Executor task launch worker for task 60 INFO Executor: Finished task 2.0 in stage 19.0 (TID 60). 2307 bytes result sent to driver 20/08/03 12:50:41.152 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 19.0 (TID 60) in 9 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:41.153 Executor task launch worker for task 61 INFO Executor: Finished task 3.0 in stage 19.0 (TID 61). 2307 bytes result sent to driver 20/08/03 12:50:41.153 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 19.0 (TID 61) in 8 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:41.153 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 19.0, whose tasks have all completed, from pool 20/08/03 12:50:41.154 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 19 (treeAggregate at RDDLossFunction.scala:61) finished in 0.034 s 20/08/03 12:50:41.154 dag-scheduler-event-loop INFO DAGScheduler: Job 19 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:41.154 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 19: Stage finished 20/08/03 12:50:41.154 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 19 finished: treeAggregate at RDDLossFunction.scala:61, took 0.038260 s 20/08/03 12:50:41.155 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(38) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:41.157 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_38_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 174.0 B, free: 2.1 GiB) 20/08/03 12:50:41.158 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO StrongWolfeLineSearch: Line search t: 0.18979597801739145 fval: 1.4032657076676387 rhs: 2.1655394799089818 cdd: 2.3029998375620355 20/08/03 12:50:41.158 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Step Size: 0.1898 20/08/03 12:50:41.158 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Val and Grad Norm: 1.40327 (rel: 0.352) 1.01078 20/08/03 12:50:41.162 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_40 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:41.165 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_40_piece0 stored as bytes in memory (estimated size 177.0 B, free 2.1 GiB) 20/08/03 12:50:41.165 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_40_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.166 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 40 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:41.173 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:41.174 dag-scheduler-event-loop INFO DAGScheduler: Got job 20 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:41.174 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 20 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:41.174 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:41.175 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:41.176 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 20 (MapPartitionsRDD[59] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:41.179 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_41 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:41.182 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_41_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:41.182 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_41_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:41.182 dag-scheduler-event-loop INFO SparkContext: Created broadcast 41 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:41.183 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 20 (MapPartitionsRDD[59] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:41.183 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 20.0 with 4 tasks 20/08/03 12:50:41.186 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 20.0 (TID 62, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.188 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 20.0 (TID 63, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.188 Executor task launch worker for task 62 INFO Executor: Running task 0.0 in stage 20.0 (TID 62) 20/08/03 12:50:41.188 Executor task launch worker for task 63 INFO Executor: Running task 1.0 in stage 20.0 (TID 63) 20/08/03 12:50:41.192 Executor task launch worker for task 62 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:41.194 Executor task launch worker for task 63 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:41.194 Executor task launch worker for task 62 INFO Executor: Finished task 0.0 in stage 20.0 (TID 62). 2307 bytes result sent to driver 20/08/03 12:50:41.196 Executor task launch worker for task 63 INFO Executor: Finished task 1.0 in stage 20.0 (TID 63). 2307 bytes result sent to driver 20/08/03 12:50:41.197 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 20.0 (TID 64, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.197 Executor task launch worker for task 64 INFO Executor: Running task 2.0 in stage 20.0 (TID 64) 20/08/03 12:50:41.200 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 20.0 (TID 65, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.200 Executor task launch worker for task 65 INFO Executor: Running task 3.0 in stage 20.0 (TID 65) 20/08/03 12:50:41.200 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 20.0 (TID 62) in 16 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:41.200 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 20.0 (TID 63) in 14 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:41.202 Executor task launch worker for task 64 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:41.204 Executor task launch worker for task 65 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:41.204 Executor task launch worker for task 64 INFO Executor: Finished task 2.0 in stage 20.0 (TID 64). 2307 bytes result sent to driver 20/08/03 12:50:41.205 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 20.0 (TID 64) in 10 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:41.206 Executor task launch worker for task 65 INFO Executor: Finished task 3.0 in stage 20.0 (TID 65). 2307 bytes result sent to driver 20/08/03 12:50:41.207 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 20.0 (TID 65) in 10 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:41.207 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 20.0, whose tasks have all completed, from pool 20/08/03 12:50:41.208 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 20 (treeAggregate at RDDLossFunction.scala:61) finished in 0.030 s 20/08/03 12:50:41.208 dag-scheduler-event-loop INFO DAGScheduler: Job 20 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:41.208 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 20: Stage finished 20/08/03 12:50:41.208 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 20 finished: treeAggregate at RDDLossFunction.scala:61, took 0.034706 s 20/08/03 12:50:41.209 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(40) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:41.211 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Step Size: 1.000 20/08/03 12:50:41.211 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Val and Grad Norm: 1.37243 (rel: 0.0220) 0.512076 20/08/03 12:50:41.211 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_40_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.214 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_42 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:41.216 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_42_piece0 stored as bytes in memory (estimated size 177.0 B, free 2.1 GiB) 20/08/03 12:50:41.217 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_42_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.217 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 42 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:41.228 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:41.229 dag-scheduler-event-loop INFO DAGScheduler: Got job 21 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:41.229 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 21 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:41.229 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:41.230 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:41.231 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 21 (MapPartitionsRDD[60] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:41.235 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_43 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:41.238 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_43_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:41.238 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_43_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:41.239 dag-scheduler-event-loop INFO SparkContext: Created broadcast 43 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:41.239 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 21 (MapPartitionsRDD[60] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:41.239 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 21.0 with 4 tasks 20/08/03 12:50:41.243 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 21.0 (TID 66, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.246 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 21.0 (TID 67, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.246 Executor task launch worker for task 67 INFO Executor: Running task 1.0 in stage 21.0 (TID 67) 20/08/03 12:50:41.246 Executor task launch worker for task 66 INFO Executor: Running task 0.0 in stage 21.0 (TID 66) 20/08/03 12:50:41.252 Executor task launch worker for task 67 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:41.253 Executor task launch worker for task 66 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:41.255 Executor task launch worker for task 67 INFO Executor: Finished task 1.0 in stage 21.0 (TID 67). 2307 bytes result sent to driver 20/08/03 12:50:41.255 Executor task launch worker for task 66 INFO Executor: Finished task 0.0 in stage 21.0 (TID 66). 2307 bytes result sent to driver 20/08/03 12:50:41.257 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 21.0 (TID 68, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.257 Executor task launch worker for task 68 INFO Executor: Running task 2.0 in stage 21.0 (TID 68) 20/08/03 12:50:41.261 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 21.0 (TID 69, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.261 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 21.0 (TID 67) in 18 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:41.261 Executor task launch worker for task 69 INFO Executor: Running task 3.0 in stage 21.0 (TID 69) 20/08/03 12:50:41.261 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 21.0 (TID 66) in 21 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:41.262 Executor task launch worker for task 68 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:41.264 Executor task launch worker for task 68 INFO Executor: Finished task 2.0 in stage 21.0 (TID 68). 2307 bytes result sent to driver 20/08/03 12:50:41.265 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 21.0 (TID 68) in 10 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:41.265 Executor task launch worker for task 69 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:41.267 Executor task launch worker for task 69 INFO Executor: Finished task 3.0 in stage 21.0 (TID 69). 2307 bytes result sent to driver 20/08/03 12:50:41.272 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 21.0 (TID 69) in 15 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:41.272 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 21.0, whose tasks have all completed, from pool 20/08/03 12:50:41.273 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 21 (treeAggregate at RDDLossFunction.scala:61) finished in 0.042 s 20/08/03 12:50:41.273 dag-scheduler-event-loop INFO DAGScheduler: Job 21 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:41.273 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 21: Stage finished 20/08/03 12:50:41.273 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 21 finished: treeAggregate at RDDLossFunction.scala:61, took 0.045419 s 20/08/03 12:50:41.274 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(42) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:41.275 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Step Size: 1.000 20/08/03 12:50:41.276 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Val and Grad Norm: 1.36307 (rel: 0.00682) 0.476493 20/08/03 12:50:41.276 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_42_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.279 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_44 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:41.281 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_44_piece0 stored as bytes in memory (estimated size 177.0 B, free 2.1 GiB) 20/08/03 12:50:41.281 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_44_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.282 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 44 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:41.293 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:41.294 dag-scheduler-event-loop INFO DAGScheduler: Got job 22 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:41.294 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 22 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:41.294 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:41.294 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:41.294 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 22 (MapPartitionsRDD[61] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:41.297 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_45 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:41.300 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_45_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:41.301 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_45_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:41.301 dag-scheduler-event-loop INFO SparkContext: Created broadcast 45 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:41.301 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 22 (MapPartitionsRDD[61] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:41.301 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 22.0 with 4 tasks 20/08/03 12:50:41.304 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 22.0 (TID 70, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.306 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 22.0 (TID 71, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.306 Executor task launch worker for task 70 INFO Executor: Running task 0.0 in stage 22.0 (TID 70) 20/08/03 12:50:41.306 Executor task launch worker for task 71 INFO Executor: Running task 1.0 in stage 22.0 (TID 71) 20/08/03 12:50:41.311 Executor task launch worker for task 71 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:41.311 Executor task launch worker for task 70 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:41.313 Executor task launch worker for task 71 INFO Executor: Finished task 1.0 in stage 22.0 (TID 71). 2307 bytes result sent to driver 20/08/03 12:50:41.313 Executor task launch worker for task 70 INFO Executor: Finished task 0.0 in stage 22.0 (TID 70). 2307 bytes result sent to driver 20/08/03 12:50:41.316 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 22.0 (TID 72, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.316 Executor task launch worker for task 72 INFO Executor: Running task 2.0 in stage 22.0 (TID 72) 20/08/03 12:50:41.319 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 22.0 (TID 73, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.320 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 22.0 (TID 71) in 16 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:41.320 Executor task launch worker for task 73 INFO Executor: Running task 3.0 in stage 22.0 (TID 73) 20/08/03 12:50:41.320 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 22.0 (TID 70) in 18 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:41.320 Executor task launch worker for task 72 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:41.322 Executor task launch worker for task 72 INFO Executor: Finished task 2.0 in stage 22.0 (TID 72). 2307 bytes result sent to driver 20/08/03 12:50:41.323 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 22.0 (TID 72) in 10 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:41.324 Executor task launch worker for task 73 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:41.327 Executor task launch worker for task 73 INFO Executor: Finished task 3.0 in stage 22.0 (TID 73). 2307 bytes result sent to driver 20/08/03 12:50:41.328 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 22.0 (TID 73) in 12 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:41.328 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 22.0, whose tasks have all completed, from pool 20/08/03 12:50:41.329 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 22 (treeAggregate at RDDLossFunction.scala:61) finished in 0.033 s 20/08/03 12:50:41.329 dag-scheduler-event-loop INFO DAGScheduler: Job 22 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:41.329 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 22: Stage finished 20/08/03 12:50:41.330 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 22 finished: treeAggregate at RDDLossFunction.scala:61, took 0.036688 s 20/08/03 12:50:41.331 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(44) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:41.332 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Step Size: 1.000 20/08/03 12:50:41.333 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Val and Grad Norm: 1.36008 (rel: 0.00219) 0.466391 20/08/03 12:50:41.333 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_44_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.337 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_46 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:41.339 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_46_piece0 stored as bytes in memory (estimated size 177.0 B, free 2.1 GiB) 20/08/03 12:50:41.339 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_46_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.340 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 46 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:41.348 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:41.349 dag-scheduler-event-loop INFO DAGScheduler: Got job 23 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:41.349 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 23 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:41.349 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:41.349 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:41.350 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 23 (MapPartitionsRDD[62] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:41.354 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_47 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:41.356 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_47_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:41.357 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_47_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:41.357 dag-scheduler-event-loop INFO SparkContext: Created broadcast 47 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:41.358 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 23 (MapPartitionsRDD[62] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:41.358 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 23.0 with 4 tasks 20/08/03 12:50:41.361 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 23.0 (TID 74, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.363 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 23.0 (TID 75, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.363 Executor task launch worker for task 75 INFO Executor: Running task 1.0 in stage 23.0 (TID 75) 20/08/03 12:50:41.363 Executor task launch worker for task 74 INFO Executor: Running task 0.0 in stage 23.0 (TID 74) 20/08/03 12:50:41.369 Executor task launch worker for task 74 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:41.369 Executor task launch worker for task 75 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:41.371 Executor task launch worker for task 74 INFO Executor: Finished task 0.0 in stage 23.0 (TID 74). 2307 bytes result sent to driver 20/08/03 12:50:41.371 Executor task launch worker for task 75 INFO Executor: Finished task 1.0 in stage 23.0 (TID 75). 2307 bytes result sent to driver 20/08/03 12:50:41.373 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 23.0 (TID 76, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.374 Executor task launch worker for task 76 INFO Executor: Running task 2.0 in stage 23.0 (TID 76) 20/08/03 12:50:41.376 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 23.0 (TID 77, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.377 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 23.0 (TID 74) in 17 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:41.377 Executor task launch worker for task 77 INFO Executor: Running task 3.0 in stage 23.0 (TID 77) 20/08/03 12:50:41.377 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 23.0 (TID 75) in 16 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:41.379 Executor task launch worker for task 76 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:41.382 Executor task launch worker for task 76 INFO Executor: Finished task 2.0 in stage 23.0 (TID 76). 2307 bytes result sent to driver 20/08/03 12:50:41.382 Executor task launch worker for task 77 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:41.383 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 23.0 (TID 76) in 11 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:41.384 Executor task launch worker for task 77 INFO Executor: Finished task 3.0 in stage 23.0 (TID 77). 2307 bytes result sent to driver 20/08/03 12:50:41.385 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 23.0 (TID 77) in 11 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:41.385 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 23.0, whose tasks have all completed, from pool 20/08/03 12:50:41.385 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 23 (treeAggregate at RDDLossFunction.scala:61) finished in 0.034 s 20/08/03 12:50:41.386 dag-scheduler-event-loop INFO DAGScheduler: Job 23 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:41.386 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 23: Stage finished 20/08/03 12:50:41.386 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 23 finished: treeAggregate at RDDLossFunction.scala:61, took 0.037847 s 20/08/03 12:50:41.387 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(46) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:41.387 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Step Size: 1.000 20/08/03 12:50:41.389 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Val and Grad Norm: 1.35917 (rel: 0.000670) 0.489978 20/08/03 12:50:41.389 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_46_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.393 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_48 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:41.396 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_48_piece0 stored as bytes in memory (estimated size 177.0 B, free 2.1 GiB) 20/08/03 12:50:41.397 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_48_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.397 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 48 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:41.405 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:41.406 dag-scheduler-event-loop INFO DAGScheduler: Got job 24 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:41.406 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 24 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:41.406 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:41.407 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:41.407 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 24 (MapPartitionsRDD[63] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:41.410 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_49 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:41.412 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_49_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:41.412 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_49_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:41.413 dag-scheduler-event-loop INFO SparkContext: Created broadcast 49 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:41.413 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 24 (MapPartitionsRDD[63] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:41.413 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 24.0 with 4 tasks 20/08/03 12:50:41.416 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 24.0 (TID 78, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.418 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 24.0 (TID 79, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.418 Executor task launch worker for task 79 INFO Executor: Running task 1.0 in stage 24.0 (TID 79) 20/08/03 12:50:41.418 Executor task launch worker for task 78 INFO Executor: Running task 0.0 in stage 24.0 (TID 78) 20/08/03 12:50:41.423 Executor task launch worker for task 78 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:41.423 Executor task launch worker for task 79 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:41.425 Executor task launch worker for task 78 INFO Executor: Finished task 0.0 in stage 24.0 (TID 78). 2307 bytes result sent to driver 20/08/03 12:50:41.425 Executor task launch worker for task 79 INFO Executor: Finished task 1.0 in stage 24.0 (TID 79). 2307 bytes result sent to driver 20/08/03 12:50:41.428 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 24.0 (TID 80, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.428 Executor task launch worker for task 80 INFO Executor: Running task 2.0 in stage 24.0 (TID 80) 20/08/03 12:50:41.430 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 24.0 (TID 81, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.430 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 24.0 (TID 78) in 16 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:41.430 Executor task launch worker for task 81 INFO Executor: Running task 3.0 in stage 24.0 (TID 81) 20/08/03 12:50:41.430 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 24.0 (TID 79) in 14 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:41.432 Executor task launch worker for task 80 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:41.434 Executor task launch worker for task 80 INFO Executor: Finished task 2.0 in stage 24.0 (TID 80). 2307 bytes result sent to driver 20/08/03 12:50:41.434 Executor task launch worker for task 81 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:41.434 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 24.0 (TID 80) in 8 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:41.436 Executor task launch worker for task 81 INFO Executor: Finished task 3.0 in stage 24.0 (TID 81). 2307 bytes result sent to driver 20/08/03 12:50:41.436 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 24.0 (TID 81) in 8 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:41.436 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 24.0, whose tasks have all completed, from pool 20/08/03 12:50:41.437 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 24 (treeAggregate at RDDLossFunction.scala:61) finished in 0.029 s 20/08/03 12:50:41.437 dag-scheduler-event-loop INFO DAGScheduler: Job 24 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:41.437 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 24: Stage finished 20/08/03 12:50:41.437 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 24 finished: treeAggregate at RDDLossFunction.scala:61, took 0.031695 s 20/08/03 12:50:41.438 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(48) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:41.439 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Step Size: 1.000 20/08/03 12:50:41.440 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Val and Grad Norm: 1.35635 (rel: 0.00207) 0.597980 20/08/03 12:50:41.440 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_48_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.443 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_50 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:41.445 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_50_piece0 stored as bytes in memory (estimated size 177.0 B, free 2.1 GiB) 20/08/03 12:50:41.446 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_50_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.446 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 50 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:41.454 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:41.455 dag-scheduler-event-loop INFO DAGScheduler: Got job 25 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:41.455 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 25 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:41.455 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:41.455 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:41.456 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 25 (MapPartitionsRDD[64] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:41.460 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_51 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:41.462 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_51_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:41.462 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_51_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:41.463 dag-scheduler-event-loop INFO SparkContext: Created broadcast 51 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:41.463 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 25 (MapPartitionsRDD[64] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:41.463 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 25.0 with 4 tasks 20/08/03 12:50:41.467 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 25.0 (TID 82, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.469 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 25.0 (TID 83, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.469 Executor task launch worker for task 82 INFO Executor: Running task 0.0 in stage 25.0 (TID 82) 20/08/03 12:50:41.469 Executor task launch worker for task 83 INFO Executor: Running task 1.0 in stage 25.0 (TID 83) 20/08/03 12:50:41.474 Executor task launch worker for task 83 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:41.475 Executor task launch worker for task 82 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:41.476 Executor task launch worker for task 83 INFO Executor: Finished task 1.0 in stage 25.0 (TID 83). 2307 bytes result sent to driver 20/08/03 12:50:41.478 Executor task launch worker for task 82 INFO Executor: Finished task 0.0 in stage 25.0 (TID 82). 2307 bytes result sent to driver 20/08/03 12:50:41.480 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 25.0 (TID 84, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.480 Executor task launch worker for task 84 INFO Executor: Running task 2.0 in stage 25.0 (TID 84) 20/08/03 12:50:41.485 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 25.0 (TID 85, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.485 Executor task launch worker for task 84 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:41.486 Executor task launch worker for task 85 INFO Executor: Running task 3.0 in stage 25.0 (TID 85) 20/08/03 12:50:41.486 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 25.0 (TID 83) in 19 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:41.486 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 25.0 (TID 82) in 22 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:41.488 Executor task launch worker for task 84 INFO Executor: Finished task 2.0 in stage 25.0 (TID 84). 2307 bytes result sent to driver 20/08/03 12:50:41.489 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 25.0 (TID 84) in 12 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:41.494 Executor task launch worker for task 85 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:41.496 Executor task launch worker for task 85 INFO Executor: Finished task 3.0 in stage 25.0 (TID 85). 2307 bytes result sent to driver 20/08/03 12:50:41.497 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 25.0 (TID 85) in 16 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:41.497 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 25.0, whose tasks have all completed, from pool 20/08/03 12:50:41.498 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 25 (treeAggregate at RDDLossFunction.scala:61) finished in 0.041 s 20/08/03 12:50:41.498 dag-scheduler-event-loop INFO DAGScheduler: Job 25 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:41.498 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 25: Stage finished 20/08/03 12:50:41.499 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 25 finished: treeAggregate at RDDLossFunction.scala:61, took 0.044988 s 20/08/03 12:50:41.500 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(50) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:41.501 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Step Size: 1.000 20/08/03 12:50:41.502 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Val and Grad Norm: 1.35404 (rel: 0.00170) 0.647957 20/08/03 12:50:41.503 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_50_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.507 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_52 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:41.509 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_52_piece0 stored as bytes in memory (estimated size 177.0 B, free 2.1 GiB) 20/08/03 12:50:41.509 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_52_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.509 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 52 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:41.517 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:41.518 dag-scheduler-event-loop INFO DAGScheduler: Got job 26 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:41.518 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 26 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:41.518 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:41.518 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:41.519 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 26 (MapPartitionsRDD[65] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:41.523 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_53 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:41.525 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_53_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:41.526 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_53_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:41.526 dag-scheduler-event-loop INFO SparkContext: Created broadcast 53 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:41.526 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 26 (MapPartitionsRDD[65] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:41.526 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 26.0 with 4 tasks 20/08/03 12:50:41.529 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 26.0 (TID 86, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.531 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 26.0 (TID 87, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.531 Executor task launch worker for task 86 INFO Executor: Running task 0.0 in stage 26.0 (TID 86) 20/08/03 12:50:41.531 Executor task launch worker for task 87 INFO Executor: Running task 1.0 in stage 26.0 (TID 87) 20/08/03 12:50:41.535 Executor task launch worker for task 87 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:41.535 Executor task launch worker for task 86 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:41.538 Executor task launch worker for task 86 INFO Executor: Finished task 0.0 in stage 26.0 (TID 86). 2307 bytes result sent to driver 20/08/03 12:50:41.538 Executor task launch worker for task 87 INFO Executor: Finished task 1.0 in stage 26.0 (TID 87). 2307 bytes result sent to driver 20/08/03 12:50:41.540 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 26.0 (TID 88, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.540 Executor task launch worker for task 88 INFO Executor: Running task 2.0 in stage 26.0 (TID 88) 20/08/03 12:50:41.543 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 26.0 (TID 89, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.543 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 26.0 (TID 86) in 16 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:41.543 Executor task launch worker for task 89 INFO Executor: Running task 3.0 in stage 26.0 (TID 89) 20/08/03 12:50:41.543 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 26.0 (TID 87) in 14 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:41.545 Executor task launch worker for task 88 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:41.546 Executor task launch worker for task 88 INFO Executor: Finished task 2.0 in stage 26.0 (TID 88). 2307 bytes result sent to driver 20/08/03 12:50:41.547 Executor task launch worker for task 89 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:41.547 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 26.0 (TID 88) in 9 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:41.548 Executor task launch worker for task 89 INFO Executor: Finished task 3.0 in stage 26.0 (TID 89). 2307 bytes result sent to driver 20/08/03 12:50:41.549 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 26.0 (TID 89) in 9 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:41.549 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 26.0, whose tasks have all completed, from pool 20/08/03 12:50:41.549 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 26 (treeAggregate at RDDLossFunction.scala:61) finished in 0.029 s 20/08/03 12:50:41.549 dag-scheduler-event-loop INFO DAGScheduler: Job 26 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:41.549 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 26: Stage finished 20/08/03 12:50:41.550 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 26 finished: treeAggregate at RDDLossFunction.scala:61, took 0.032772 s 20/08/03 12:50:41.551 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(52) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:41.551 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Step Size: 1.000 20/08/03 12:50:41.552 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Val and Grad Norm: 1.34903 (rel: 0.00370) 0.657457 20/08/03 12:50:41.552 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_52_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.557 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_54 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:41.558 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_54_piece0 stored as bytes in memory (estimated size 177.0 B, free 2.1 GiB) 20/08/03 12:50:41.559 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_54_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.559 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 54 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:41.566 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:41.567 dag-scheduler-event-loop INFO DAGScheduler: Got job 27 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:41.567 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 27 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:41.567 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:41.568 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:41.568 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 27 (MapPartitionsRDD[66] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:41.570 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_55 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:41.573 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_55_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:41.573 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_55_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:41.574 dag-scheduler-event-loop INFO SparkContext: Created broadcast 55 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:41.574 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 27 (MapPartitionsRDD[66] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:41.574 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 27.0 with 4 tasks 20/08/03 12:50:41.578 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 27.0 (TID 90, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.582 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 27.0 (TID 91, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.582 Executor task launch worker for task 91 INFO Executor: Running task 1.0 in stage 27.0 (TID 91) 20/08/03 12:50:41.582 Executor task launch worker for task 90 INFO Executor: Running task 0.0 in stage 27.0 (TID 90) 20/08/03 12:50:41.586 Executor task launch worker for task 91 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:41.586 Executor task launch worker for task 90 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:41.589 Executor task launch worker for task 91 INFO Executor: Finished task 1.0 in stage 27.0 (TID 91). 2307 bytes result sent to driver 20/08/03 12:50:41.589 Executor task launch worker for task 90 INFO Executor: Finished task 0.0 in stage 27.0 (TID 90). 2307 bytes result sent to driver 20/08/03 12:50:41.592 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 27.0 (TID 92, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.592 Executor task launch worker for task 92 INFO Executor: Running task 2.0 in stage 27.0 (TID 92) 20/08/03 12:50:41.592 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 27.0 (TID 91) in 13 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:41.595 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 27.0 (TID 93, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.595 Executor task launch worker for task 93 INFO Executor: Running task 3.0 in stage 27.0 (TID 93) 20/08/03 12:50:41.595 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 27.0 (TID 90) in 20 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:41.597 Executor task launch worker for task 92 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:41.599 Executor task launch worker for task 92 INFO Executor: Finished task 2.0 in stage 27.0 (TID 92). 2307 bytes result sent to driver 20/08/03 12:50:41.599 Executor task launch worker for task 93 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:41.600 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 27.0 (TID 92) in 11 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:41.601 Executor task launch worker for task 93 INFO Executor: Finished task 3.0 in stage 27.0 (TID 93). 2307 bytes result sent to driver 20/08/03 12:50:41.602 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 27.0 (TID 93) in 9 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:41.602 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 27.0, whose tasks have all completed, from pool 20/08/03 12:50:41.602 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 27 (treeAggregate at RDDLossFunction.scala:61) finished in 0.033 s 20/08/03 12:50:41.602 dag-scheduler-event-loop INFO DAGScheduler: Job 27 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:41.602 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 27: Stage finished 20/08/03 12:50:41.602 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 27 finished: treeAggregate at RDDLossFunction.scala:61, took 0.035890 s 20/08/03 12:50:41.603 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(54) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:41.604 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Step Size: 1.000 20/08/03 12:50:41.605 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Val and Grad Norm: 1.34602 (rel: 0.00223) 0.502452 20/08/03 12:50:41.605 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_54_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.609 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_56 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:41.611 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_56_piece0 stored as bytes in memory (estimated size 177.0 B, free 2.1 GiB) 20/08/03 12:50:41.611 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_56_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.611 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 56 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:41.619 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:41.620 dag-scheduler-event-loop INFO DAGScheduler: Got job 28 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:41.620 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 28 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:41.620 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:41.621 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:41.622 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 28 (MapPartitionsRDD[67] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:41.625 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_57 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:41.628 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_57_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:41.628 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_57_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:41.628 dag-scheduler-event-loop INFO SparkContext: Created broadcast 57 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:41.629 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 28 (MapPartitionsRDD[67] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:41.629 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 28.0 with 4 tasks 20/08/03 12:50:41.632 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 28.0 (TID 94, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.634 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 28.0 (TID 95, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.634 Executor task launch worker for task 95 INFO Executor: Running task 1.0 in stage 28.0 (TID 95) 20/08/03 12:50:41.634 Executor task launch worker for task 94 INFO Executor: Running task 0.0 in stage 28.0 (TID 94) 20/08/03 12:50:41.638 Executor task launch worker for task 95 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:41.638 Executor task launch worker for task 94 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:41.640 Executor task launch worker for task 95 INFO Executor: Finished task 1.0 in stage 28.0 (TID 95). 2307 bytes result sent to driver 20/08/03 12:50:41.640 Executor task launch worker for task 94 INFO Executor: Finished task 0.0 in stage 28.0 (TID 94). 2307 bytes result sent to driver 20/08/03 12:50:41.642 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 28.0 (TID 96, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.642 Executor task launch worker for task 96 INFO Executor: Running task 2.0 in stage 28.0 (TID 96) 20/08/03 12:50:41.644 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 28.0 (TID 97, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.644 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 28.0 (TID 95) in 12 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:41.644 Executor task launch worker for task 97 INFO Executor: Running task 3.0 in stage 28.0 (TID 97) 20/08/03 12:50:41.645 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 28.0 (TID 94) in 15 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:41.646 Executor task launch worker for task 96 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:41.648 Executor task launch worker for task 96 INFO Executor: Finished task 2.0 in stage 28.0 (TID 96). 2307 bytes result sent to driver 20/08/03 12:50:41.648 Executor task launch worker for task 97 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:41.648 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 28.0 (TID 96) in 8 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:41.650 Executor task launch worker for task 97 INFO Executor: Finished task 3.0 in stage 28.0 (TID 97). 2307 bytes result sent to driver 20/08/03 12:50:41.650 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 28.0 (TID 97) in 8 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:41.651 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 28.0, whose tasks have all completed, from pool 20/08/03 12:50:41.651 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 28 (treeAggregate at RDDLossFunction.scala:61) finished in 0.028 s 20/08/03 12:50:41.651 dag-scheduler-event-loop INFO DAGScheduler: Job 28 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:41.651 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 28: Stage finished 20/08/03 12:50:41.652 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 28 finished: treeAggregate at RDDLossFunction.scala:61, took 0.032734 s 20/08/03 12:50:41.652 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(56) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:41.653 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Step Size: 1.000 20/08/03 12:50:41.654 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Val and Grad Norm: 1.34596 (rel: 4.28e-05) 0.479564 20/08/03 12:50:41.654 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_56_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.659 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_58 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:41.660 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_58_piece0 stored as bytes in memory (estimated size 177.0 B, free 2.1 GiB) 20/08/03 12:50:41.660 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_58_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.660 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 58 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:41.667 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:41.668 dag-scheduler-event-loop INFO DAGScheduler: Got job 29 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:41.668 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 29 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:41.668 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:41.669 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:41.669 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 29 (MapPartitionsRDD[68] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:41.673 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_59 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:41.676 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_59_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:41.676 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_59_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:41.677 dag-scheduler-event-loop INFO SparkContext: Created broadcast 59 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:41.677 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 29 (MapPartitionsRDD[68] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:41.677 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 29.0 with 4 tasks 20/08/03 12:50:41.680 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 29.0 (TID 98, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.682 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 29.0 (TID 99, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.682 Executor task launch worker for task 98 INFO Executor: Running task 0.0 in stage 29.0 (TID 98) 20/08/03 12:50:41.682 Executor task launch worker for task 99 INFO Executor: Running task 1.0 in stage 29.0 (TID 99) 20/08/03 12:50:41.687 Executor task launch worker for task 98 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:41.687 Executor task launch worker for task 99 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:41.689 Executor task launch worker for task 98 INFO Executor: Finished task 0.0 in stage 29.0 (TID 98). 2307 bytes result sent to driver 20/08/03 12:50:41.689 Executor task launch worker for task 99 INFO Executor: Finished task 1.0 in stage 29.0 (TID 99). 2307 bytes result sent to driver 20/08/03 12:50:41.692 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 29.0 (TID 100, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.693 Executor task launch worker for task 100 INFO Executor: Running task 2.0 in stage 29.0 (TID 100) 20/08/03 12:50:41.696 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 29.0 (TID 101, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.696 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 29.0 (TID 98) in 18 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:41.696 Executor task launch worker for task 101 INFO Executor: Running task 3.0 in stage 29.0 (TID 101) 20/08/03 12:50:41.696 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 29.0 (TID 99) in 16 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:41.698 Executor task launch worker for task 100 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:41.700 Executor task launch worker for task 100 INFO Executor: Finished task 2.0 in stage 29.0 (TID 100). 2307 bytes result sent to driver 20/08/03 12:50:41.700 Executor task launch worker for task 101 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:41.700 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 29.0 (TID 100) in 11 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:41.702 Executor task launch worker for task 101 INFO Executor: Finished task 3.0 in stage 29.0 (TID 101). 2307 bytes result sent to driver 20/08/03 12:50:41.703 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 29.0 (TID 101) in 10 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:41.703 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 29.0, whose tasks have all completed, from pool 20/08/03 12:50:41.703 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 29 (treeAggregate at RDDLossFunction.scala:61) finished in 0.033 s 20/08/03 12:50:41.704 dag-scheduler-event-loop INFO DAGScheduler: Job 29 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:41.704 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 29: Stage finished 20/08/03 12:50:41.704 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 29 finished: treeAggregate at RDDLossFunction.scala:61, took 0.036420 s 20/08/03 12:50:41.705 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(58) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:41.706 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Step Size: 1.000 20/08/03 12:50:41.706 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Val and Grad Norm: 1.34596 (rel: 2.34e-07) 0.478330 20/08/03 12:50:41.706 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_58_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.710 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_60 stored as values in memory (estimated size 96.0 B, free 2.1 GiB) 20/08/03 12:50:41.711 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_60_piece0 stored as bytes in memory (estimated size 177.0 B, free 2.1 GiB) 20/08/03 12:50:41.712 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_60_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.712 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 60 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:41.720 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:41.722 dag-scheduler-event-loop INFO DAGScheduler: Got job 30 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:41.723 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 30 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:41.723 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:41.723 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:41.724 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 30 (MapPartitionsRDD[69] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:41.726 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_61 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:41.729 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_61_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:41.730 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_61_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:41.730 dag-scheduler-event-loop INFO SparkContext: Created broadcast 61 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:41.731 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 30 (MapPartitionsRDD[69] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:41.731 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 30.0 with 4 tasks 20/08/03 12:50:41.733 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 30.0 (TID 102, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.735 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 30.0 (TID 103, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.735 Executor task launch worker for task 102 INFO Executor: Running task 0.0 in stage 30.0 (TID 102) 20/08/03 12:50:41.736 Executor task launch worker for task 103 INFO Executor: Running task 1.0 in stage 30.0 (TID 103) 20/08/03 12:50:41.740 Executor task launch worker for task 102 INFO BlockManager: Found block rdd_42_0 locally 20/08/03 12:50:41.740 Executor task launch worker for task 103 INFO BlockManager: Found block rdd_42_1 locally 20/08/03 12:50:41.742 Executor task launch worker for task 103 INFO Executor: Finished task 1.0 in stage 30.0 (TID 103). 2307 bytes result sent to driver 20/08/03 12:50:41.742 Executor task launch worker for task 102 INFO Executor: Finished task 0.0 in stage 30.0 (TID 102). 2307 bytes result sent to driver 20/08/03 12:50:41.744 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 30.0 (TID 104, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.744 Executor task launch worker for task 104 INFO Executor: Running task 2.0 in stage 30.0 (TID 104) 20/08/03 12:50:41.747 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 30.0 (TID 105, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:41.748 Executor task launch worker for task 105 INFO Executor: Running task 3.0 in stage 30.0 (TID 105) 20/08/03 12:50:41.748 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 30.0 (TID 103) in 14 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:41.748 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 30.0 (TID 102) in 16 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:41.749 Executor task launch worker for task 104 INFO BlockManager: Found block rdd_42_2 locally 20/08/03 12:50:41.751 Executor task launch worker for task 104 INFO Executor: Finished task 2.0 in stage 30.0 (TID 104). 2307 bytes result sent to driver 20/08/03 12:50:41.751 Executor task launch worker for task 105 INFO BlockManager: Found block rdd_42_3 locally 20/08/03 12:50:41.752 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 30.0 (TID 104) in 10 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:41.753 Executor task launch worker for task 105 INFO Executor: Finished task 3.0 in stage 30.0 (TID 105). 2307 bytes result sent to driver 20/08/03 12:50:41.753 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 30.0 (TID 105) in 9 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:41.753 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 30.0, whose tasks have all completed, from pool 20/08/03 12:50:41.754 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 30 (treeAggregate at RDDLossFunction.scala:61) finished in 0.030 s 20/08/03 12:50:41.754 dag-scheduler-event-loop INFO DAGScheduler: Job 30 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:41.754 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 30: Stage finished 20/08/03 12:50:41.754 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 30 finished: treeAggregate at RDDLossFunction.scala:61, took 0.034392 s 20/08/03 12:50:41.755 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(60) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:41.755 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Step Size: 1.000 20/08/03 12:50:41.756 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Val and Grad Norm: 1.34596 (rel: 4.64e-09) 0.478265 20/08/03 12:50:41.757 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_60_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 177.0 B, free: 2.1 GiB) 20/08/03 12:50:41.758 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGSB: Converged because projected step converged 20/08/03 12:50:41.758 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(19) (from destroy at LogisticRegression.scala:804) 20/08/03 12:50:41.759 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MapPartitionsRDD: Removing RDD 42 from persistence list 20/08/03 12:50:41.759 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_19_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 101.0 B, free: 2.1 GiB) 20/08/03 12:50:41.760 block-manager-slave-async-thread-pool-86 INFO BlockManager: Removing RDD 42 20/08/03 12:50:41.964 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 21.840704 ms 20/08/03 12:50:42.064 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 23.283418 ms 20/08/03 12:50:42.073 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [24c64493] training finished 20/08/03 12:50:42.137 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [38b00bcd] Stage class: LogisticRegression 20/08/03 12:50:42.137 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [38b00bcd] Stage uid: logreg_2482145152d0 20/08/03 12:50:42.181 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [38b00bcd] training: numPartitions=4 storageLevel=StorageLevel(1 replicas) 20/08/03 12:50:42.181 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [38b00bcd] {} 20/08/03 12:50:42.191 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at LogisticRegression.scala:499 20/08/03 12:50:42.192 dag-scheduler-event-loop INFO DAGScheduler: Got job 31 (treeAggregate at LogisticRegression.scala:499) with 4 output partitions 20/08/03 12:50:42.192 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 31 (treeAggregate at LogisticRegression.scala:499) 20/08/03 12:50:42.192 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:42.192 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:42.193 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 31 (MapPartitionsRDD[94] at treeAggregate at LogisticRegression.scala:499), which has no missing parents 20/08/03 12:50:42.199 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_62 stored as values in memory (estimated size 27.5 KiB, free 2.1 GiB) 20/08/03 12:50:42.202 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_62_piece0 stored as bytes in memory (estimated size 10.7 KiB, free 2.1 GiB) 20/08/03 12:50:42.202 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_62_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.7 KiB, free: 2.1 GiB) 20/08/03 12:50:42.202 dag-scheduler-event-loop INFO SparkContext: Created broadcast 62 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:42.203 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 31 (MapPartitionsRDD[94] at treeAggregate at LogisticRegression.scala:499) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:42.203 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 31.0 with 4 tasks 20/08/03 12:50:42.206 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 31.0 (TID 106, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:42.208 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 31.0 (TID 107, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:42.208 Executor task launch worker for task 106 INFO Executor: Running task 0.0 in stage 31.0 (TID 106) 20/08/03 12:50:42.208 Executor task launch worker for task 107 INFO Executor: Running task 1.0 in stage 31.0 (TID 107) 20/08/03 12:50:42.243 Executor task launch worker for task 106 INFO MemoryStore: Block rdd_88_0 stored as values in memory (estimated size 244.2 KiB, free 2.1 GiB) 20/08/03 12:50:42.243 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_88_0 in memory on amp-jenkins-worker-06.amp:34426 (size: 244.2 KiB, free: 2.1 GiB) 20/08/03 12:50:42.247 Executor task launch worker for task 106 INFO Executor: Finished task 0.0 in stage 31.0 (TID 106). 2687 bytes result sent to driver 20/08/03 12:50:42.249 Executor task launch worker for task 107 INFO MemoryStore: Block rdd_88_1 stored as values in memory (estimated size 244.2 KiB, free 2.1 GiB) 20/08/03 12:50:42.250 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 31.0 (TID 108, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:42.250 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_88_1 in memory on amp-jenkins-worker-06.amp:34426 (size: 244.2 KiB, free: 2.1 GiB) 20/08/03 12:50:42.250 Executor task launch worker for task 108 INFO Executor: Running task 2.0 in stage 31.0 (TID 108) 20/08/03 12:50:42.250 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 31.0 (TID 106) in 46 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:42.253 Executor task launch worker for task 107 INFO Executor: Finished task 1.0 in stage 31.0 (TID 107). 2687 bytes result sent to driver 20/08/03 12:50:42.256 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 31.0 (TID 109, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:42.256 Executor task launch worker for task 109 INFO Executor: Running task 3.0 in stage 31.0 (TID 109) 20/08/03 12:50:42.256 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 31.0 (TID 107) in 50 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:42.288 Executor task launch worker for task 108 INFO MemoryStore: Block rdd_88_2 stored as values in memory (estimated size 244.2 KiB, free 2.1 GiB) 20/08/03 12:50:42.289 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_88_2 in memory on amp-jenkins-worker-06.amp:34426 (size: 244.2 KiB, free: 2.1 GiB) 20/08/03 12:50:42.290 Executor task launch worker for task 109 INFO MemoryStore: Block rdd_88_3 stored as values in memory (estimated size 244.2 KiB, free 2.1 GiB) 20/08/03 12:50:42.291 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_88_3 in memory on amp-jenkins-worker-06.amp:34426 (size: 244.2 KiB, free: 2.1 GiB) 20/08/03 12:50:42.293 Executor task launch worker for task 108 INFO Executor: Finished task 2.0 in stage 31.0 (TID 108). 2687 bytes result sent to driver 20/08/03 12:50:42.294 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 31.0 (TID 108) in 46 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:42.295 Executor task launch worker for task 109 INFO Executor: Finished task 3.0 in stage 31.0 (TID 109). 2687 bytes result sent to driver 20/08/03 12:50:42.296 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 31.0 (TID 109) in 43 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:42.296 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 31.0, whose tasks have all completed, from pool 20/08/03 12:50:42.297 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 31 (treeAggregate at LogisticRegression.scala:499) finished in 0.103 s 20/08/03 12:50:42.298 dag-scheduler-event-loop INFO DAGScheduler: Job 31 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:42.298 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 31: Stage finished 20/08/03 12:50:42.298 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 31 finished: treeAggregate at LogisticRegression.scala:499, took 0.107742 s 20/08/03 12:50:42.299 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [38b00bcd] {"numExamples":10000} 20/08/03 12:50:42.299 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [38b00bcd] {"lowestLabelWeight":"4383.0"} 20/08/03 12:50:42.299 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [38b00bcd] {"highestLabelWeight":"5617.0"} 20/08/03 12:50:42.300 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [38b00bcd] {"sumOfWeights":10000.0} 20/08/03 12:50:42.306 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite ERROR Instrumentation: java.lang.IllegalArgumentException: requirement failed: LowerBoundsOnCoefficients should always be less than or equal to upperBoundsOnCoefficients, but found: lowerBoundsOnCoefficients = 1.0 0.0 1.0 0.0 , upperBoundsOnCoefficients = 0.0 1.0 1.0 0.0 . at scala.Predef$.require(Predef.scala:281) at org.apache.spark.ml.classification.LogisticRegression.assertBoundConstrainedOptimizationParamsValid(LogisticRegression.scala:461) at org.apache.spark.ml.classification.LogisticRegression.$anonfun$train$1(LogisticRegression.scala:539) at org.apache.spark.ml.util.Instrumentation$.$anonfun$instrumented$1(Instrumentation.scala:191) at scala.util.Try$.apply(Try.scala:213) at org.apache.spark.ml.util.Instrumentation$.instrumented(Instrumentation.scala:191) at org.apache.spark.ml.classification.LogisticRegression.train(LogisticRegression.scala:487) at org.apache.spark.ml.classification.LogisticRegression.train(LogisticRegression.scala:482) at org.apache.spark.ml.classification.LogisticRegression.train(LogisticRegression.scala:281) at org.apache.spark.ml.Predictor.fit(Predictor.scala:150) at org.apache.spark.ml.classification.LogisticRegressionSuite.$anonfun$new$11(LogisticRegressionSuite.scala:209) at org.scalatest.Assertions.intercept(Assertions.scala:807) at org.scalatest.Assertions.intercept$(Assertions.scala:804) at org.scalatest.FunSuite.intercept(FunSuite.scala:1560) at org.apache.spark.ml.classification.LogisticRegressionSuite.$anonfun$new$10(LogisticRegressionSuite.scala:205) at org.scalatest.Assertions.withClue(Assertions.scala:1223) at org.scalatest.Assertions.withClue$(Assertions.scala:1210) at org.scalatest.FunSuite.withClue(FunSuite.scala:1560) at org.apache.spark.ml.classification.LogisticRegressionSuite.$anonfun$new$7(LogisticRegressionSuite.scala:205) at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85) at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83) at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) at org.scalatest.Transformer.apply(Transformer.scala:22) at org.scalatest.Transformer.apply(Transformer.scala:20) at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:186) at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:157) at org.scalatest.FunSuiteLike.invokeWithFixture$1(FunSuiteLike.scala:184) at org.scalatest.FunSuiteLike.$anonfun$runTest$1(FunSuiteLike.scala:196) at org.scalatest.SuperEngine.runTestImpl(Engine.scala:286) at org.scalatest.FunSuiteLike.runTest(FunSuiteLike.scala:196) at org.scalatest.FunSuiteLike.runTest$(FunSuiteLike.scala:178) at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:59) at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:221) at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:214) at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:59) at org.scalatest.FunSuiteLike.$anonfun$runTests$1(FunSuiteLike.scala:229) at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:393) at scala.collection.immutable.List.foreach(List.scala:392) at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:381) at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:376) at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:458) at org.scalatest.FunSuiteLike.runTests(FunSuiteLike.scala:229) at org.scalatest.FunSuiteLike.runTests$(FunSuiteLike.scala:228) at org.scalatest.FunSuite.runTests(FunSuite.scala:1560) at org.scalatest.Suite.run(Suite.scala:1124) at org.scalatest.Suite.run$(Suite.scala:1106) at org.scalatest.FunSuite.org$scalatest$FunSuiteLike$$super$run(FunSuite.scala:1560) at org.scalatest.FunSuiteLike.$anonfun$run$1(FunSuiteLike.scala:233) at org.scalatest.SuperEngine.runImpl(Engine.scala:518) at org.scalatest.FunSuiteLike.run(FunSuiteLike.scala:233) at org.scalatest.FunSuiteLike.run$(FunSuiteLike.scala:232) at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:59) at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213) at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210) at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208) at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:59) at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:317) at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:510) at sbt.ForkMain$Run$2.call(ForkMain.java:296) at sbt.ForkMain$Run$2.call(ForkMain.java:286) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 20/08/03 12:50:42.369 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [4abb59eb] Stage class: LogisticRegression 20/08/03 12:50:42.520 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [4abb59eb] Stage uid: logreg_43f370bc8bb3 20/08/03 12:50:42.560 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [4abb59eb] training: numPartitions=4 storageLevel=StorageLevel(1 replicas) 20/08/03 12:50:42.560 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [4abb59eb] {} 20/08/03 12:50:42.569 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at LogisticRegression.scala:499 20/08/03 12:50:42.570 dag-scheduler-event-loop INFO DAGScheduler: Got job 32 (treeAggregate at LogisticRegression.scala:499) with 4 output partitions 20/08/03 12:50:42.570 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 32 (treeAggregate at LogisticRegression.scala:499) 20/08/03 12:50:42.570 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:42.571 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:42.572 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 32 (MapPartitionsRDD[106] at treeAggregate at LogisticRegression.scala:499), which has no missing parents 20/08/03 12:50:42.579 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_63 stored as values in memory (estimated size 27.5 KiB, free 2.1 GiB) 20/08/03 12:50:42.581 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_63_piece0 stored as bytes in memory (estimated size 10.8 KiB, free 2.1 GiB) 20/08/03 12:50:42.582 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_63_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.8 KiB, free: 2.1 GiB) 20/08/03 12:50:42.582 dag-scheduler-event-loop INFO SparkContext: Created broadcast 63 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:42.582 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 32 (MapPartitionsRDD[106] at treeAggregate at LogisticRegression.scala:499) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:42.582 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 32.0 with 4 tasks 20/08/03 12:50:42.585 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 32.0 (TID 110, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:42.587 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 32.0 (TID 111, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:42.587 Executor task launch worker for task 110 INFO Executor: Running task 0.0 in stage 32.0 (TID 110) 20/08/03 12:50:42.587 Executor task launch worker for task 111 INFO Executor: Running task 1.0 in stage 32.0 (TID 111) 20/08/03 12:50:42.620 Executor task launch worker for task 111 INFO MemoryStore: Block rdd_100_1 stored as values in memory (estimated size 244.2 KiB, free 2.1 GiB) 20/08/03 12:50:42.620 Executor task launch worker for task 110 INFO MemoryStore: Block rdd_100_0 stored as values in memory (estimated size 244.2 KiB, free 2.1 GiB) 20/08/03 12:50:43.392 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_100_1 in memory on amp-jenkins-worker-06.amp:34426 (size: 244.2 KiB, free: 2.1 GiB) 20/08/03 12:50:43.393 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_100_0 in memory on amp-jenkins-worker-06.amp:34426 (size: 244.2 KiB, free: 2.1 GiB) 20/08/03 12:50:43.398 Executor task launch worker for task 111 INFO Executor: Finished task 1.0 in stage 32.0 (TID 111). 2687 bytes result sent to driver 20/08/03 12:50:43.398 Executor task launch worker for task 110 INFO Executor: Finished task 0.0 in stage 32.0 (TID 110). 2687 bytes result sent to driver 20/08/03 12:50:43.405 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 32.0 (TID 112, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:43.405 Executor task launch worker for task 112 INFO Executor: Running task 2.0 in stage 32.0 (TID 112) 20/08/03 12:50:43.408 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 32.0 (TID 113, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 162557 bytes) 20/08/03 12:50:43.408 Executor task launch worker for task 113 INFO Executor: Running task 3.0 in stage 32.0 (TID 113) 20/08/03 12:50:43.408 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 32.0 (TID 111) in 823 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:43.409 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 32.0 (TID 110) in 826 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:43.438 Executor task launch worker for task 112 INFO MemoryStore: Block rdd_100_2 stored as values in memory (estimated size 244.2 KiB, free 2.1 GiB) 20/08/03 12:50:43.439 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_100_2 in memory on amp-jenkins-worker-06.amp:34426 (size: 244.2 KiB, free: 2.1 GiB) 20/08/03 12:50:43.442 Executor task launch worker for task 112 INFO Executor: Finished task 2.0 in stage 32.0 (TID 112). 2687 bytes result sent to driver 20/08/03 12:50:43.443 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 32.0 (TID 112) in 45 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:43.450 Executor task launch worker for task 113 INFO MemoryStore: Block rdd_100_3 stored as values in memory (estimated size 244.2 KiB, free 2.1 GiB) 20/08/03 12:50:43.451 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_100_3 in memory on amp-jenkins-worker-06.amp:34426 (size: 244.2 KiB, free: 2.1 GiB) 20/08/03 12:50:43.456 Executor task launch worker for task 113 INFO Executor: Finished task 3.0 in stage 32.0 (TID 113). 2687 bytes result sent to driver 20/08/03 12:50:43.457 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 32.0 (TID 113) in 51 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:43.457 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 32.0, whose tasks have all completed, from pool 20/08/03 12:50:43.458 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 32 (treeAggregate at LogisticRegression.scala:499) finished in 0.885 s 20/08/03 12:50:43.458 dag-scheduler-event-loop INFO DAGScheduler: Job 32 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:43.458 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 32: Stage finished 20/08/03 12:50:43.458 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 32 finished: treeAggregate at LogisticRegression.scala:499, took 0.889291 s 20/08/03 12:50:43.459 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [4abb59eb] {"numExamples":10000} 20/08/03 12:50:43.460 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [4abb59eb] {"lowestLabelWeight":"4383.0"} 20/08/03 12:50:43.460 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [4abb59eb] {"highestLabelWeight":"5617.0"} 20/08/03 12:50:43.460 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [4abb59eb] {"sumOfWeights":10000.0} 20/08/03 12:50:43.461 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite ERROR Instrumentation: java.lang.IllegalArgumentException: requirement failed: The shape of upperBoundsOnCoefficients must be compatible with (1, number of features) for binomial regression, or (number of classes, number of features) for multinomial regression, but found: (1, 3). at scala.Predef$.require(Predef.scala:281) at org.apache.spark.ml.classification.LogisticRegression.assertBoundConstrainedOptimizationParamsValid(LogisticRegression.scala:444) at org.apache.spark.ml.classification.LogisticRegression.$anonfun$train$1(LogisticRegression.scala:539) at org.apache.spark.ml.util.Instrumentation$.$anonfun$instrumented$1(Instrumentation.scala:191) at scala.util.Try$.apply(Try.scala:213) at org.apache.spark.ml.util.Instrumentation$.instrumented(Instrumentation.scala:191) at org.apache.spark.ml.classification.LogisticRegression.train(LogisticRegression.scala:487) at org.apache.spark.ml.classification.LogisticRegression.train(LogisticRegression.scala:482) at org.apache.spark.ml.classification.LogisticRegression.train(LogisticRegression.scala:281) at org.apache.spark.ml.Predictor.fit(Predictor.scala:150) at org.apache.spark.ml.classification.LogisticRegressionSuite.$anonfun$new$13(LogisticRegressionSuite.scala:218) at org.scalatest.Assertions.intercept(Assertions.scala:807) at org.scalatest.Assertions.intercept$(Assertions.scala:804) at org.scalatest.FunSuite.intercept(FunSuite.scala:1560) at org.apache.spark.ml.classification.LogisticRegressionSuite.$anonfun$new$12(LogisticRegressionSuite.scala:214) at org.scalatest.Assertions.withClue(Assertions.scala:1223) at org.scalatest.Assertions.withClue$(Assertions.scala:1210) at org.scalatest.FunSuite.withClue(FunSuite.scala:1560) at org.apache.spark.ml.classification.LogisticRegressionSuite.$anonfun$new$7(LogisticRegressionSuite.scala:214) at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85) at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83) at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) at org.scalatest.Transformer.apply(Transformer.scala:22) at org.scalatest.Transformer.apply(Transformer.scala:20) at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:186) at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:157) at org.scalatest.FunSuiteLike.invokeWithFixture$1(FunSuiteLike.scala:184) at org.scalatest.FunSuiteLike.$anonfun$runTest$1(FunSuiteLike.scala:196) at org.scalatest.SuperEngine.runTestImpl(Engine.scala:286) at org.scalatest.FunSuiteLike.runTest(FunSuiteLike.scala:196) at org.scalatest.FunSuiteLike.runTest$(FunSuiteLike.scala:178) at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:59) at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:221) at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:214) at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:59) at org.scalatest.FunSuiteLike.$anonfun$runTests$1(FunSuiteLike.scala:229) at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:393) at scala.collection.immutable.List.foreach(List.scala:392) at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:381) at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:376) at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:458) at org.scalatest.FunSuiteLike.runTests(FunSuiteLike.scala:229) at org.scalatest.FunSuiteLike.runTests$(FunSuiteLike.scala:228) at org.scalatest.FunSuite.runTests(FunSuite.scala:1560) at org.scalatest.Suite.run(Suite.scala:1124) at org.scalatest.Suite.run$(Suite.scala:1106) at org.scalatest.FunSuite.org$scalatest$FunSuiteLike$$super$run(FunSuite.scala:1560) at org.scalatest.FunSuiteLike.$anonfun$run$1(FunSuiteLike.scala:233) at org.scalatest.SuperEngine.runImpl(Engine.scala:518) at org.scalatest.FunSuiteLike.run(FunSuiteLike.scala:233) at org.scalatest.FunSuiteLike.run$(FunSuiteLike.scala:232) at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:59) at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213) at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210) at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208) at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:59) at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:317) at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:510) at sbt.ForkMain$Run$2.call(ForkMain.java:296) at sbt.ForkMain$Run$2.call(ForkMain.java:286) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 20/08/03 12:50:43.465 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: ===== FINISHED o.a.s.ml.classification.LogisticRegressionSuite: 'logistic regression: illegal params' ===== 20/08/03 12:50:43.466 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: ===== TEST OUTPUT FOR o.a.s.ml.classification.LogisticRegressionSuite: 'empty probabilityCol or predictionCol' ===== 20/08/03 12:50:43.507 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [b32afdbb] Stage class: LogisticRegression 20/08/03 12:50:43.507 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [b32afdbb] Stage uid: logreg_02444d550004 20/08/03 12:50:43.533 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [b32afdbb] training: numPartitions=2 storageLevel=StorageLevel(1 replicas) 20/08/03 12:50:43.606 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [b32afdbb] {"predictionCol":"prediction","probabilityCol":"","maxIter":1} 20/08/03 12:50:43.618 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at LogisticRegression.scala:499 20/08/03 12:50:43.620 dag-scheduler-event-loop INFO DAGScheduler: Got job 33 (treeAggregate at LogisticRegression.scala:499) with 2 output partitions 20/08/03 12:50:43.621 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 33 (treeAggregate at LogisticRegression.scala:499) 20/08/03 12:50:43.621 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:43.622 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:43.623 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 33 (MapPartitionsRDD[118] at treeAggregate at LogisticRegression.scala:499), which has no missing parents 20/08/03 12:50:43.632 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_64 stored as values in memory (estimated size 12.6 KiB, free 2.1 GiB) 20/08/03 12:50:43.634 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_64_piece0 stored as bytes in memory (estimated size 6.4 KiB, free 2.1 GiB) 20/08/03 12:50:43.635 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_64_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 6.4 KiB, free: 2.1 GiB) 20/08/03 12:50:43.635 dag-scheduler-event-loop INFO SparkContext: Created broadcast 64 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:43.636 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 33 (MapPartitionsRDD[118] at treeAggregate at LogisticRegression.scala:499) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:43.636 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 33.0 with 2 tasks 20/08/03 12:50:43.637 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 33.0 (TID 114, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:43.638 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 33.0 (TID 115, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:43.638 Executor task launch worker for task 115 INFO Executor: Running task 1.0 in stage 33.0 (TID 115) 20/08/03 12:50:43.638 Executor task launch worker for task 114 INFO Executor: Running task 0.0 in stage 33.0 (TID 114) 20/08/03 12:50:43.646 Executor task launch worker for task 115 INFO MemoryStore: Block rdd_112_1 stored as values in memory (estimated size 3.7 KiB, free 2.1 GiB) 20/08/03 12:50:43.646 Executor task launch worker for task 114 INFO MemoryStore: Block rdd_112_0 stored as values in memory (estimated size 3.7 KiB, free 2.1 GiB) 20/08/03 12:50:43.646 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_112_1 in memory on amp-jenkins-worker-06.amp:34426 (size: 3.7 KiB, free: 2.1 GiB) 20/08/03 12:50:43.646 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_112_0 in memory on amp-jenkins-worker-06.amp:34426 (size: 3.7 KiB, free: 2.1 GiB) 20/08/03 12:50:43.660 Executor task launch worker for task 115 INFO Executor: Finished task 1.0 in stage 33.0 (TID 115). 2543 bytes result sent to driver 20/08/03 12:50:43.660 Executor task launch worker for task 114 INFO Executor: Finished task 0.0 in stage 33.0 (TID 114). 2629 bytes result sent to driver 20/08/03 12:50:43.661 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 33.0 (TID 115) in 23 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:43.661 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 33.0 (TID 114) in 24 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:43.662 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 33.0, whose tasks have all completed, from pool 20/08/03 12:50:43.662 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_49_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:43.662 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 33 (treeAggregate at LogisticRegression.scala:499) finished in 0.038 s 20/08/03 12:50:43.663 dag-scheduler-event-loop INFO DAGScheduler: Job 33 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:43.663 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 33: Stage finished 20/08/03 12:50:43.663 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 33 finished: treeAggregate at LogisticRegression.scala:499, took 0.044184 s 20/08/03 12:50:43.664 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [b32afdbb] {"numExamples":100} 20/08/03 12:50:43.664 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [b32afdbb] {"lowestLabelWeight":"34.0"} 20/08/03 12:50:43.664 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [b32afdbb] {"highestLabelWeight":"66.0"} 20/08/03 12:50:43.664 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [b32afdbb] {"sumOfWeights":100.0} 20/08/03 12:50:43.665 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [b32afdbb] {"numClasses":2} 20/08/03 12:50:43.665 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [b32afdbb] {"numFeatures":1} 20/08/03 12:50:43.666 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_65 stored as values in memory (estimated size 48.0 B, free 2.1 GiB) 20/08/03 12:50:43.666 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_59_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:43.668 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_57_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:43.668 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_65_piece0 stored as bytes in memory (estimated size 77.0 B, free 2.1 GiB) 20/08/03 12:50:43.668 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_65_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 77.0 B, free: 2.1 GiB) 20/08/03 12:50:43.669 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 65 from broadcast at LogisticRegression.scala:593 20/08/03 12:50:43.669 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_61_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:43.670 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_66 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:43.670 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_45_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:43.673 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_37_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:43.673 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_66_piece0 stored as bytes in memory (estimated size 153.0 B, free 2.1 GiB) 20/08/03 12:50:43.673 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_66_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 153.0 B, free: 2.1 GiB) 20/08/03 12:50:43.673 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 66 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:43.673 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_55_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:43.675 block-manager-slave-async-thread-pool-29 INFO BlockManager: Removing RDD 42 20/08/03 12:50:43.677 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_43_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:43.678 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_51_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:43.679 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_35_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:43.680 block-manager-slave-async-thread-pool-48 INFO BlockManager: Removing RDD 88 20/08/03 12:50:43.681 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:43.682 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_33_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:43.682 dag-scheduler-event-loop INFO DAGScheduler: Got job 34 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:43.682 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 34 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:43.682 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:43.682 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:43.682 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_62_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.7 KiB, free: 2.1 GiB) 20/08/03 12:50:43.683 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 34 (MapPartitionsRDD[119] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:43.685 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_67 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:43.685 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_39_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:43.686 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_47_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:43.687 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_53_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:43.687 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_67_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:43.687 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_67_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:43.687 dag-scheduler-event-loop INFO SparkContext: Created broadcast 67 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:43.688 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 34 (MapPartitionsRDD[119] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:43.688 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 34.0 with 2 tasks 20/08/03 12:50:43.688 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_41_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:43.689 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 34.0 (TID 116, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:43.689 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_63_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.8 KiB, free: 2.1 GiB) 20/08/03 12:50:43.689 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 34.0 (TID 117, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:43.689 Executor task launch worker for task 117 INFO Executor: Running task 1.0 in stage 34.0 (TID 117) 20/08/03 12:50:43.689 Executor task launch worker for task 116 INFO Executor: Running task 0.0 in stage 34.0 (TID 116) 20/08/03 12:50:43.690 block-manager-slave-async-thread-pool-94 INFO BlockManager: Removing RDD 100 20/08/03 12:50:43.691 Executor task launch worker for task 116 INFO BlockManager: Found block rdd_112_0 locally 20/08/03 12:50:43.691 Executor task launch worker for task 117 INFO BlockManager: Found block rdd_112_1 locally 20/08/03 12:50:43.692 Executor task launch worker for task 116 INFO Executor: Finished task 0.0 in stage 34.0 (TID 116). 2168 bytes result sent to driver 20/08/03 12:50:43.692 Executor task launch worker for task 117 INFO Executor: Finished task 1.0 in stage 34.0 (TID 117). 2168 bytes result sent to driver 20/08/03 12:50:43.693 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 34.0 (TID 116) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:43.693 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 34.0 (TID 117) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:43.693 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 34.0, whose tasks have all completed, from pool 20/08/03 12:50:43.693 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 34 (treeAggregate at RDDLossFunction.scala:61) finished in 0.009 s 20/08/03 12:50:43.693 dag-scheduler-event-loop INFO DAGScheduler: Job 34 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:43.693 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 34: Stage finished 20/08/03 12:50:43.694 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 34 finished: treeAggregate at RDDLossFunction.scala:61, took 0.012052 s 20/08/03 12:50:43.694 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(66) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:43.695 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_66_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 153.0 B, free: 2.1 GiB) 20/08/03 12:50:43.695 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_68 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:43.697 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_68_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:43.697 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_68_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:43.698 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 68 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:43.705 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:43.706 dag-scheduler-event-loop INFO DAGScheduler: Got job 35 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:43.706 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 35 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:43.706 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:43.706 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:43.707 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 35 (MapPartitionsRDD[120] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:43.709 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_69 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:43.711 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_69_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:43.711 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_69_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:43.712 dag-scheduler-event-loop INFO SparkContext: Created broadcast 69 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:43.712 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 35 (MapPartitionsRDD[120] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:43.712 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 35.0 with 2 tasks 20/08/03 12:50:43.713 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 35.0 (TID 118, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:43.713 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 35.0 (TID 119, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:43.713 Executor task launch worker for task 118 INFO Executor: Running task 0.0 in stage 35.0 (TID 118) 20/08/03 12:50:43.713 Executor task launch worker for task 119 INFO Executor: Running task 1.0 in stage 35.0 (TID 119) 20/08/03 12:50:43.716 Executor task launch worker for task 118 INFO BlockManager: Found block rdd_112_0 locally 20/08/03 12:50:43.716 Executor task launch worker for task 119 INFO BlockManager: Found block rdd_112_1 locally 20/08/03 12:50:43.716 Executor task launch worker for task 118 INFO Executor: Finished task 0.0 in stage 35.0 (TID 118). 2168 bytes result sent to driver 20/08/03 12:50:43.717 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 35.0 (TID 118) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:43.717 Executor task launch worker for task 119 INFO Executor: Finished task 1.0 in stage 35.0 (TID 119). 2168 bytes result sent to driver 20/08/03 12:50:43.717 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 35.0 (TID 119) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:43.717 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 35.0, whose tasks have all completed, from pool 20/08/03 12:50:43.717 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 35 (treeAggregate at RDDLossFunction.scala:61) finished in 0.009 s 20/08/03 12:50:43.718 dag-scheduler-event-loop INFO DAGScheduler: Job 35 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:43.718 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 35: Stage finished 20/08/03 12:50:43.718 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 35 finished: treeAggregate at RDDLossFunction.scala:61, took 0.012684 s 20/08/03 12:50:43.718 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(68) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:43.719 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 3.941 20/08/03 12:50:43.719 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.493391 (rel: 0.230) 0.0621158 20/08/03 12:50:43.719 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_68_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:43.720 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Converged because max iterations reached 20/08/03 12:50:43.720 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(65) (from destroy at LogisticRegression.scala:804) 20/08/03 12:50:43.721 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MapPartitionsRDD: Removing RDD 112 from persistence list 20/08/03 12:50:43.721 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_65_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 77.0 B, free: 2.1 GiB) 20/08/03 12:50:43.722 block-manager-slave-async-thread-pool-7 INFO BlockManager: Removing RDD 112 20/08/03 12:50:43.825 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [b32afdbb] training finished 20/08/03 12:50:43.864 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5224ec4a] Stage class: LogisticRegression 20/08/03 12:50:43.864 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5224ec4a] Stage uid: logreg_02444d550004 20/08/03 12:50:43.888 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5224ec4a] training: numPartitions=2 storageLevel=StorageLevel(1 replicas) 20/08/03 12:50:43.889 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5224ec4a] {"predictionCol":"","probabilityCol":"probability","maxIter":1} 20/08/03 12:50:43.897 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at LogisticRegression.scala:499 20/08/03 12:50:43.898 dag-scheduler-event-loop INFO DAGScheduler: Got job 36 (treeAggregate at LogisticRegression.scala:499) with 2 output partitions 20/08/03 12:50:43.898 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 36 (treeAggregate at LogisticRegression.scala:499) 20/08/03 12:50:43.898 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:43.898 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:43.900 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 36 (MapPartitionsRDD[145] at treeAggregate at LogisticRegression.scala:499), which has no missing parents 20/08/03 12:50:43.904 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_70 stored as values in memory (estimated size 12.6 KiB, free 2.1 GiB) 20/08/03 12:50:43.907 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_70_piece0 stored as bytes in memory (estimated size 6.4 KiB, free 2.1 GiB) 20/08/03 12:50:43.907 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_70_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 6.4 KiB, free: 2.1 GiB) 20/08/03 12:50:43.908 dag-scheduler-event-loop INFO SparkContext: Created broadcast 70 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:43.908 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 36 (MapPartitionsRDD[145] at treeAggregate at LogisticRegression.scala:499) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:43.908 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 36.0 with 2 tasks 20/08/03 12:50:43.909 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 36.0 (TID 120, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:43.909 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 36.0 (TID 121, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:43.909 Executor task launch worker for task 120 INFO Executor: Running task 0.0 in stage 36.0 (TID 120) 20/08/03 12:50:43.909 Executor task launch worker for task 121 INFO Executor: Running task 1.0 in stage 36.0 (TID 121) 20/08/03 12:50:43.915 Executor task launch worker for task 120 INFO MemoryStore: Block rdd_139_0 stored as values in memory (estimated size 3.7 KiB, free 2.1 GiB) 20/08/03 12:50:43.915 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_139_0 in memory on amp-jenkins-worker-06.amp:34426 (size: 3.7 KiB, free: 2.1 GiB) 20/08/03 12:50:43.916 Executor task launch worker for task 121 INFO MemoryStore: Block rdd_139_1 stored as values in memory (estimated size 3.7 KiB, free 2.1 GiB) 20/08/03 12:50:43.916 Executor task launch worker for task 120 INFO Executor: Finished task 0.0 in stage 36.0 (TID 120). 2543 bytes result sent to driver 20/08/03 12:50:43.916 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_139_1 in memory on amp-jenkins-worker-06.amp:34426 (size: 3.7 KiB, free: 2.1 GiB) 20/08/03 12:50:43.917 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 36.0 (TID 120) in 8 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:43.917 Executor task launch worker for task 121 INFO Executor: Finished task 1.0 in stage 36.0 (TID 121). 2543 bytes result sent to driver 20/08/03 12:50:43.918 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 36.0 (TID 121) in 9 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:43.918 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 36.0, whose tasks have all completed, from pool 20/08/03 12:50:43.918 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 36 (treeAggregate at LogisticRegression.scala:499) finished in 0.018 s 20/08/03 12:50:43.918 dag-scheduler-event-loop INFO DAGScheduler: Job 36 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:43.918 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 36: Stage finished 20/08/03 12:50:43.919 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 36 finished: treeAggregate at LogisticRegression.scala:499, took 0.021422 s 20/08/03 12:50:43.919 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5224ec4a] {"numExamples":100} 20/08/03 12:50:43.919 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5224ec4a] {"lowestLabelWeight":"34.0"} 20/08/03 12:50:43.919 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5224ec4a] {"highestLabelWeight":"66.0"} 20/08/03 12:50:43.919 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5224ec4a] {"sumOfWeights":100.0} 20/08/03 12:50:43.920 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5224ec4a] {"numClasses":2} 20/08/03 12:50:43.920 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5224ec4a] {"numFeatures":1} 20/08/03 12:50:43.920 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_71 stored as values in memory (estimated size 48.0 B, free 2.1 GiB) 20/08/03 12:50:43.922 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_71_piece0 stored as bytes in memory (estimated size 77.0 B, free 2.1 GiB) 20/08/03 12:50:43.922 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_71_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 77.0 B, free: 2.1 GiB) 20/08/03 12:50:43.923 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 71 from broadcast at LogisticRegression.scala:593 20/08/03 12:50:43.923 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_72 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:43.924 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_72_piece0 stored as bytes in memory (estimated size 153.0 B, free 2.1 GiB) 20/08/03 12:50:43.924 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_72_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 153.0 B, free: 2.1 GiB) 20/08/03 12:50:43.925 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 72 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:43.932 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:43.933 dag-scheduler-event-loop INFO DAGScheduler: Got job 37 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:43.933 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 37 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:43.933 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:43.933 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:43.933 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 37 (MapPartitionsRDD[146] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:43.937 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_73 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:43.939 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_73_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:43.939 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_73_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:43.939 dag-scheduler-event-loop INFO SparkContext: Created broadcast 73 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:43.939 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 37 (MapPartitionsRDD[146] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:43.939 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 37.0 with 2 tasks 20/08/03 12:50:43.940 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 37.0 (TID 122, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:43.941 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 37.0 (TID 123, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:43.941 Executor task launch worker for task 122 INFO Executor: Running task 0.0 in stage 37.0 (TID 122) 20/08/03 12:50:43.941 Executor task launch worker for task 123 INFO Executor: Running task 1.0 in stage 37.0 (TID 123) 20/08/03 12:50:43.943 Executor task launch worker for task 123 INFO BlockManager: Found block rdd_139_1 locally 20/08/03 12:50:43.943 Executor task launch worker for task 122 INFO BlockManager: Found block rdd_139_0 locally 20/08/03 12:50:43.943 Executor task launch worker for task 123 INFO Executor: Finished task 1.0 in stage 37.0 (TID 123). 2168 bytes result sent to driver 20/08/03 12:50:43.944 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 37.0 (TID 123) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:43.944 Executor task launch worker for task 122 INFO Executor: Finished task 0.0 in stage 37.0 (TID 122). 2168 bytes result sent to driver 20/08/03 12:50:43.944 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 37.0 (TID 122) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:43.944 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 37.0, whose tasks have all completed, from pool 20/08/03 12:50:43.944 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 37 (treeAggregate at RDDLossFunction.scala:61) finished in 0.008 s 20/08/03 12:50:43.945 dag-scheduler-event-loop INFO DAGScheduler: Job 37 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:43.945 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 37: Stage finished 20/08/03 12:50:43.945 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 37 finished: treeAggregate at RDDLossFunction.scala:61, took 0.012607 s 20/08/03 12:50:43.945 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(72) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:43.946 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_72_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 153.0 B, free: 2.1 GiB) 20/08/03 12:50:43.947 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_74 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:43.948 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_74_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:43.948 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_74_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:43.949 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 74 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:43.956 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:43.956 dag-scheduler-event-loop INFO DAGScheduler: Got job 38 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:43.956 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 38 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:43.956 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:43.957 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:43.957 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 38 (MapPartitionsRDD[147] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:43.958 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_75 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:43.959 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_75_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:43.960 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_75_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:43.960 dag-scheduler-event-loop INFO SparkContext: Created broadcast 75 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:43.960 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 38 (MapPartitionsRDD[147] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:43.960 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 38.0 with 2 tasks 20/08/03 12:50:43.961 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 38.0 (TID 124, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:43.961 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 38.0 (TID 125, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:43.961 Executor task launch worker for task 124 INFO Executor: Running task 0.0 in stage 38.0 (TID 124) 20/08/03 12:50:43.961 Executor task launch worker for task 125 INFO Executor: Running task 1.0 in stage 38.0 (TID 125) 20/08/03 12:50:43.963 Executor task launch worker for task 125 INFO BlockManager: Found block rdd_139_1 locally 20/08/03 12:50:43.963 Executor task launch worker for task 124 INFO BlockManager: Found block rdd_139_0 locally 20/08/03 12:50:43.964 Executor task launch worker for task 125 INFO Executor: Finished task 1.0 in stage 38.0 (TID 125). 2168 bytes result sent to driver 20/08/03 12:50:43.964 Executor task launch worker for task 124 INFO Executor: Finished task 0.0 in stage 38.0 (TID 124). 2168 bytes result sent to driver 20/08/03 12:50:43.964 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 38.0 (TID 125) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:43.964 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 38.0 (TID 124) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:43.964 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 38.0, whose tasks have all completed, from pool 20/08/03 12:50:43.965 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 38 (treeAggregate at RDDLossFunction.scala:61) finished in 0.007 s 20/08/03 12:50:43.965 dag-scheduler-event-loop INFO DAGScheduler: Job 38 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:43.965 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 38: Stage finished 20/08/03 12:50:43.965 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 38 finished: treeAggregate at RDDLossFunction.scala:61, took 0.008938 s 20/08/03 12:50:43.965 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(74) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:43.966 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 3.941 20/08/03 12:50:43.966 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.493391 (rel: 0.230) 0.0621158 20/08/03 12:50:43.966 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_74_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:43.966 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Converged because max iterations reached 20/08/03 12:50:43.966 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(71) (from destroy at LogisticRegression.scala:804) 20/08/03 12:50:43.967 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MapPartitionsRDD: Removing RDD 139 from persistence list 20/08/03 12:50:43.967 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_71_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 77.0 B, free: 2.1 GiB) 20/08/03 12:50:43.967 block-manager-slave-async-thread-pool-14 INFO BlockManager: Removing RDD 139 20/08/03 12:50:44.057 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5224ec4a] training finished 20/08/03 12:50:44.092 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5c30fdaf] Stage class: LogisticRegression 20/08/03 12:50:44.092 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5c30fdaf] Stage uid: logreg_02444d550004 20/08/03 12:50:44.116 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5c30fdaf] training: numPartitions=2 storageLevel=StorageLevel(1 replicas) 20/08/03 12:50:44.116 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5c30fdaf] {"predictionCol":"","probabilityCol":"","maxIter":1} 20/08/03 12:50:44.125 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at LogisticRegression.scala:499 20/08/03 12:50:44.126 dag-scheduler-event-loop INFO DAGScheduler: Got job 39 (treeAggregate at LogisticRegression.scala:499) with 2 output partitions 20/08/03 12:50:44.126 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 39 (treeAggregate at LogisticRegression.scala:499) 20/08/03 12:50:44.126 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:44.126 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:44.127 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 39 (MapPartitionsRDD[172] at treeAggregate at LogisticRegression.scala:499), which has no missing parents 20/08/03 12:50:44.132 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_76 stored as values in memory (estimated size 12.6 KiB, free 2.1 GiB) 20/08/03 12:50:44.134 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_76_piece0 stored as bytes in memory (estimated size 6.4 KiB, free 2.1 GiB) 20/08/03 12:50:44.134 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_76_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 6.4 KiB, free: 2.1 GiB) 20/08/03 12:50:44.134 dag-scheduler-event-loop INFO SparkContext: Created broadcast 76 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:44.138 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 39 (MapPartitionsRDD[172] at treeAggregate at LogisticRegression.scala:499) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:44.138 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 39.0 with 2 tasks 20/08/03 12:50:44.139 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 39.0 (TID 126, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:44.139 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 39.0 (TID 127, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:44.140 Executor task launch worker for task 126 INFO Executor: Running task 0.0 in stage 39.0 (TID 126) 20/08/03 12:50:44.140 Executor task launch worker for task 127 INFO Executor: Running task 1.0 in stage 39.0 (TID 127) 20/08/03 12:50:44.145 Executor task launch worker for task 127 INFO MemoryStore: Block rdd_166_1 stored as values in memory (estimated size 3.7 KiB, free 2.1 GiB) 20/08/03 12:50:44.146 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_166_1 in memory on amp-jenkins-worker-06.amp:34426 (size: 3.7 KiB, free: 2.1 GiB) 20/08/03 12:50:44.146 Executor task launch worker for task 126 INFO MemoryStore: Block rdd_166_0 stored as values in memory (estimated size 3.7 KiB, free 2.1 GiB) 20/08/03 12:50:44.147 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_166_0 in memory on amp-jenkins-worker-06.amp:34426 (size: 3.7 KiB, free: 2.1 GiB) 20/08/03 12:50:44.147 Executor task launch worker for task 127 INFO Executor: Finished task 1.0 in stage 39.0 (TID 127). 2543 bytes result sent to driver 20/08/03 12:50:44.147 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 39.0 (TID 127) in 8 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:44.148 Executor task launch worker for task 126 INFO Executor: Finished task 0.0 in stage 39.0 (TID 126). 2543 bytes result sent to driver 20/08/03 12:50:44.148 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 39.0 (TID 126) in 9 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:44.148 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 39.0, whose tasks have all completed, from pool 20/08/03 12:50:44.149 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 39 (treeAggregate at LogisticRegression.scala:499) finished in 0.022 s 20/08/03 12:50:44.149 dag-scheduler-event-loop INFO DAGScheduler: Job 39 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:44.149 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 39: Stage finished 20/08/03 12:50:44.150 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 39 finished: treeAggregate at LogisticRegression.scala:499, took 0.024484 s 20/08/03 12:50:44.150 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5c30fdaf] {"numExamples":100} 20/08/03 12:50:44.150 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5c30fdaf] {"lowestLabelWeight":"34.0"} 20/08/03 12:50:44.150 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5c30fdaf] {"highestLabelWeight":"66.0"} 20/08/03 12:50:44.150 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5c30fdaf] {"sumOfWeights":100.0} 20/08/03 12:50:44.151 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5c30fdaf] {"numClasses":2} 20/08/03 12:50:44.151 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5c30fdaf] {"numFeatures":1} 20/08/03 12:50:44.151 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_77 stored as values in memory (estimated size 48.0 B, free 2.1 GiB) 20/08/03 12:50:44.153 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_77_piece0 stored as bytes in memory (estimated size 77.0 B, free 2.1 GiB) 20/08/03 12:50:44.153 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_77_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 77.0 B, free: 2.1 GiB) 20/08/03 12:50:44.154 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 77 from broadcast at LogisticRegression.scala:593 20/08/03 12:50:44.154 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_78 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:44.156 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_78_piece0 stored as bytes in memory (estimated size 153.0 B, free 2.1 GiB) 20/08/03 12:50:44.156 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_78_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 153.0 B, free: 2.1 GiB) 20/08/03 12:50:44.156 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 78 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:44.164 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:44.164 dag-scheduler-event-loop INFO DAGScheduler: Got job 40 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:44.165 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 40 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:44.165 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:44.165 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:44.165 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 40 (MapPartitionsRDD[173] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:44.167 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_79 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:44.169 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_79_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:44.169 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_79_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:44.169 dag-scheduler-event-loop INFO SparkContext: Created broadcast 79 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:44.169 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 40 (MapPartitionsRDD[173] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:44.169 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 40.0 with 2 tasks 20/08/03 12:50:44.170 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 40.0 (TID 128, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:44.171 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 40.0 (TID 129, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:44.171 Executor task launch worker for task 129 INFO Executor: Running task 1.0 in stage 40.0 (TID 129) 20/08/03 12:50:44.171 Executor task launch worker for task 128 INFO Executor: Running task 0.0 in stage 40.0 (TID 128) 20/08/03 12:50:44.175 Executor task launch worker for task 128 INFO BlockManager: Found block rdd_166_0 locally 20/08/03 12:50:44.175 Executor task launch worker for task 129 INFO BlockManager: Found block rdd_166_1 locally 20/08/03 12:50:44.176 Executor task launch worker for task 129 INFO Executor: Finished task 1.0 in stage 40.0 (TID 129). 2211 bytes result sent to driver 20/08/03 12:50:44.176 Executor task launch worker for task 128 INFO Executor: Finished task 0.0 in stage 40.0 (TID 128). 2168 bytes result sent to driver 20/08/03 12:50:44.177 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 40.0 (TID 129) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:44.177 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 40.0 (TID 128) in 7 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:44.177 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 40.0, whose tasks have all completed, from pool 20/08/03 12:50:44.177 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 40 (treeAggregate at RDDLossFunction.scala:61) finished in 0.011 s 20/08/03 12:50:44.177 dag-scheduler-event-loop INFO DAGScheduler: Job 40 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:44.177 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 40: Stage finished 20/08/03 12:50:44.178 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 40 finished: treeAggregate at RDDLossFunction.scala:61, took 0.013701 s 20/08/03 12:50:44.178 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(78) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:44.179 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_78_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 153.0 B, free: 2.1 GiB) 20/08/03 12:50:44.180 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_80 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:44.181 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_80_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:44.181 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_80_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:44.181 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 80 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:44.191 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:44.192 dag-scheduler-event-loop INFO DAGScheduler: Got job 41 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:44.192 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 41 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:44.192 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:44.192 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:44.192 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 41 (MapPartitionsRDD[174] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:44.194 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_81 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:44.196 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_81_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:44.196 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_81_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:44.196 dag-scheduler-event-loop INFO SparkContext: Created broadcast 81 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:44.196 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 41 (MapPartitionsRDD[174] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:44.196 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 41.0 with 2 tasks 20/08/03 12:50:44.197 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 41.0 (TID 130, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:44.198 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 41.0 (TID 131, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:44.198 Executor task launch worker for task 130 INFO Executor: Running task 0.0 in stage 41.0 (TID 130) 20/08/03 12:50:44.198 Executor task launch worker for task 131 INFO Executor: Running task 1.0 in stage 41.0 (TID 131) 20/08/03 12:50:44.200 Executor task launch worker for task 130 INFO BlockManager: Found block rdd_166_0 locally 20/08/03 12:50:44.201 Executor task launch worker for task 131 INFO BlockManager: Found block rdd_166_1 locally 20/08/03 12:50:44.201 Executor task launch worker for task 130 INFO Executor: Finished task 0.0 in stage 41.0 (TID 130). 2168 bytes result sent to driver 20/08/03 12:50:44.201 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 41.0 (TID 130) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:44.201 Executor task launch worker for task 131 INFO Executor: Finished task 1.0 in stage 41.0 (TID 131). 2168 bytes result sent to driver 20/08/03 12:50:44.202 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 41.0 (TID 131) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:44.202 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 41.0, whose tasks have all completed, from pool 20/08/03 12:50:44.202 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 41 (treeAggregate at RDDLossFunction.scala:61) finished in 0.009 s 20/08/03 12:50:44.202 dag-scheduler-event-loop INFO DAGScheduler: Job 41 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:44.202 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 41: Stage finished 20/08/03 12:50:44.203 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 41 finished: treeAggregate at RDDLossFunction.scala:61, took 0.011443 s 20/08/03 12:50:44.203 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(80) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:44.204 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 3.941 20/08/03 12:50:44.204 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_80_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:44.204 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.493391 (rel: 0.230) 0.0621158 20/08/03 12:50:44.204 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Converged because max iterations reached 20/08/03 12:50:44.204 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(77) (from destroy at LogisticRegression.scala:804) 20/08/03 12:50:44.205 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MapPartitionsRDD: Removing RDD 166 from persistence list 20/08/03 12:50:44.205 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_77_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 77.0 B, free: 2.1 GiB) 20/08/03 12:50:44.205 block-manager-slave-async-thread-pool-31 INFO BlockManager: Removing RDD 166 20/08/03 12:50:44.316 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5c30fdaf] training finished 20/08/03 12:50:44.368 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [af4636b4] Stage class: LogisticRegression 20/08/03 12:50:44.368 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [af4636b4] Stage uid: logreg_02444d550004 20/08/03 12:50:44.408 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 15.462334 ms 20/08/03 12:50:44.414 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [af4636b4] training: numPartitions=4 storageLevel=StorageLevel(1 replicas) 20/08/03 12:50:44.414 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [af4636b4] {"predictionCol":"prediction","probabilityCol":"","maxIter":1} 20/08/03 12:50:44.422 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at LogisticRegression.scala:499 20/08/03 12:50:44.423 dag-scheduler-event-loop INFO DAGScheduler: Got job 42 (treeAggregate at LogisticRegression.scala:499) with 4 output partitions 20/08/03 12:50:44.423 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 42 (treeAggregate at LogisticRegression.scala:499) 20/08/03 12:50:44.423 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:44.424 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:44.424 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 42 (MapPartitionsRDD[199] at treeAggregate at LogisticRegression.scala:499), which has no missing parents 20/08/03 12:50:44.435 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_82 stored as values in memory (estimated size 27.5 KiB, free 2.1 GiB) 20/08/03 12:50:44.438 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_82_piece0 stored as bytes in memory (estimated size 10.7 KiB, free 2.1 GiB) 20/08/03 12:50:44.438 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_82_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.7 KiB, free: 2.1 GiB) 20/08/03 12:50:44.439 dag-scheduler-event-loop INFO SparkContext: Created broadcast 82 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:44.439 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 42 (MapPartitionsRDD[199] at treeAggregate at LogisticRegression.scala:499) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:44.439 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 42.0 with 4 tasks 20/08/03 12:50:44.440 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 42.0 (TID 132, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.440 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 42.0 (TID 133, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.441 Executor task launch worker for task 132 INFO Executor: Running task 0.0 in stage 42.0 (TID 132) 20/08/03 12:50:44.441 Executor task launch worker for task 133 INFO Executor: Running task 1.0 in stage 42.0 (TID 133) 20/08/03 12:50:44.446 Executor task launch worker for task 132 INFO MemoryStore: Block rdd_193_0 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:44.446 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_193_0 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:44.447 Executor task launch worker for task 133 INFO MemoryStore: Block rdd_193_1 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:44.447 Executor task launch worker for task 132 INFO Executor: Finished task 0.0 in stage 42.0 (TID 132). 2673 bytes result sent to driver 20/08/03 12:50:44.447 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_193_1 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:44.448 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 42.0 (TID 134, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.448 Executor task launch worker for task 134 INFO Executor: Running task 2.0 in stage 42.0 (TID 134) 20/08/03 12:50:44.448 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 42.0 (TID 132) in 8 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:44.448 Executor task launch worker for task 133 INFO Executor: Finished task 1.0 in stage 42.0 (TID 133). 2639 bytes result sent to driver 20/08/03 12:50:44.449 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 42.0 (TID 135, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.449 Executor task launch worker for task 135 INFO Executor: Running task 3.0 in stage 42.0 (TID 135) 20/08/03 12:50:44.449 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 42.0 (TID 133) in 9 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:44.452 Executor task launch worker for task 134 INFO MemoryStore: Block rdd_193_2 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:44.452 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_193_2 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:44.453 Executor task launch worker for task 134 INFO Executor: Finished task 2.0 in stage 42.0 (TID 134). 2673 bytes result sent to driver 20/08/03 12:50:44.454 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 42.0 (TID 134) in 7 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:44.455 Executor task launch worker for task 135 INFO MemoryStore: Block rdd_193_3 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:44.455 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_193_3 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:44.456 Executor task launch worker for task 135 INFO Executor: Finished task 3.0 in stage 42.0 (TID 135). 2639 bytes result sent to driver 20/08/03 12:50:44.456 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 42.0 (TID 135) in 7 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:44.456 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 42.0, whose tasks have all completed, from pool 20/08/03 12:50:44.456 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 42 (treeAggregate at LogisticRegression.scala:499) finished in 0.031 s 20/08/03 12:50:44.457 dag-scheduler-event-loop INFO DAGScheduler: Job 42 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:44.457 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 42: Stage finished 20/08/03 12:50:44.457 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 42 finished: treeAggregate at LogisticRegression.scala:499, took 0.034635 s 20/08/03 12:50:44.457 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [af4636b4] {"numExamples":100} 20/08/03 12:50:44.458 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [af4636b4] {"lowestLabelWeight":"3.0"} 20/08/03 12:50:44.458 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [af4636b4] {"highestLabelWeight":"67.0"} 20/08/03 12:50:44.458 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [af4636b4] {"sumOfWeights":100.0} 20/08/03 12:50:44.458 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [af4636b4] {"numClasses":3} 20/08/03 12:50:44.458 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [af4636b4] {"numFeatures":2} 20/08/03 12:50:44.458 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_83 stored as values in memory (estimated size 56.0 B, free 2.1 GiB) 20/08/03 12:50:44.460 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_83_piece0 stored as bytes in memory (estimated size 85.0 B, free 2.1 GiB) 20/08/03 12:50:44.460 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_83_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 85.0 B, free: 2.1 GiB) 20/08/03 12:50:44.460 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 83 from broadcast at LogisticRegression.scala:593 20/08/03 12:50:44.463 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_84 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:44.463 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_84_piece0 stored as bytes in memory (estimated size 171.0 B, free 2.1 GiB) 20/08/03 12:50:44.464 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_84_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 171.0 B, free: 2.1 GiB) 20/08/03 12:50:44.464 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 84 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:44.471 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:44.472 dag-scheduler-event-loop INFO DAGScheduler: Got job 43 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:44.472 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 43 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:44.472 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:44.472 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:44.472 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 43 (MapPartitionsRDD[200] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:44.474 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_85 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:44.476 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_85_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:44.477 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_85_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:44.477 dag-scheduler-event-loop INFO SparkContext: Created broadcast 85 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:44.477 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 43 (MapPartitionsRDD[200] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:44.477 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 43.0 with 4 tasks 20/08/03 12:50:44.478 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 43.0 (TID 136, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.478 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 43.0 (TID 137, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.479 Executor task launch worker for task 136 INFO Executor: Running task 0.0 in stage 43.0 (TID 136) 20/08/03 12:50:44.479 Executor task launch worker for task 137 INFO Executor: Running task 1.0 in stage 43.0 (TID 137) 20/08/03 12:50:44.481 Executor task launch worker for task 136 INFO BlockManager: Found block rdd_193_0 locally 20/08/03 12:50:44.482 Executor task launch worker for task 137 INFO BlockManager: Found block rdd_193_1 locally 20/08/03 12:50:44.488 Executor task launch worker for task 136 INFO Executor: Finished task 0.0 in stage 43.0 (TID 136). 2339 bytes result sent to driver 20/08/03 12:50:44.488 Executor task launch worker for task 137 INFO Executor: Finished task 1.0 in stage 43.0 (TID 137). 2339 bytes result sent to driver 20/08/03 12:50:44.488 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 43.0 (TID 138, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.488 Executor task launch worker for task 138 INFO Executor: Running task 2.0 in stage 43.0 (TID 138) 20/08/03 12:50:44.488 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 43.0 (TID 139, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.489 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 43.0 (TID 136) in 11 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:44.489 Executor task launch worker for task 139 INFO Executor: Running task 3.0 in stage 43.0 (TID 139) 20/08/03 12:50:44.489 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 43.0 (TID 137) in 11 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:44.491 Executor task launch worker for task 139 INFO BlockManager: Found block rdd_193_3 locally 20/08/03 12:50:44.491 Executor task launch worker for task 138 INFO BlockManager: Found block rdd_193_2 locally 20/08/03 12:50:44.492 Executor task launch worker for task 139 INFO Executor: Finished task 3.0 in stage 43.0 (TID 139). 2296 bytes result sent to driver 20/08/03 12:50:44.492 task-result-getter-2 INFO TaskSetManager: Finished task 3.0 in stage 43.0 (TID 139) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:44.492 Executor task launch worker for task 138 INFO Executor: Finished task 2.0 in stage 43.0 (TID 138). 2339 bytes result sent to driver 20/08/03 12:50:44.493 task-result-getter-3 INFO TaskSetManager: Finished task 2.0 in stage 43.0 (TID 138) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:44.493 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 43.0, whose tasks have all completed, from pool 20/08/03 12:50:44.493 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 43 (treeAggregate at RDDLossFunction.scala:61) finished in 0.020 s 20/08/03 12:50:44.493 dag-scheduler-event-loop INFO DAGScheduler: Job 43 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:44.493 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 43: Stage finished 20/08/03 12:50:44.494 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 43 finished: treeAggregate at RDDLossFunction.scala:61, took 0.022153 s 20/08/03 12:50:44.494 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(84) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:44.495 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_84_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 171.0 B, free: 2.1 GiB) 20/08/03 12:50:44.495 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_86 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:44.496 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_86_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:44.497 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_86_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:44.497 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 86 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:44.504 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:44.505 dag-scheduler-event-loop INFO DAGScheduler: Got job 44 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:44.505 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 44 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:44.505 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:44.505 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:44.505 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 44 (MapPartitionsRDD[201] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:44.507 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_87 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:44.510 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_87_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:44.510 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_87_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:44.510 dag-scheduler-event-loop INFO SparkContext: Created broadcast 87 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:44.511 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 44 (MapPartitionsRDD[201] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:44.511 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 44.0 with 4 tasks 20/08/03 12:50:44.512 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 44.0 (TID 140, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.512 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 44.0 (TID 141, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.512 Executor task launch worker for task 141 INFO Executor: Running task 1.0 in stage 44.0 (TID 141) 20/08/03 12:50:44.512 Executor task launch worker for task 140 INFO Executor: Running task 0.0 in stage 44.0 (TID 140) 20/08/03 12:50:44.515 Executor task launch worker for task 141 INFO BlockManager: Found block rdd_193_1 locally 20/08/03 12:50:44.515 Executor task launch worker for task 140 INFO BlockManager: Found block rdd_193_0 locally 20/08/03 12:50:44.516 Executor task launch worker for task 140 INFO Executor: Finished task 0.0 in stage 44.0 (TID 140). 2339 bytes result sent to driver 20/08/03 12:50:44.516 Executor task launch worker for task 141 INFO Executor: Finished task 1.0 in stage 44.0 (TID 141). 2339 bytes result sent to driver 20/08/03 12:50:44.516 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 44.0 (TID 142, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.516 Executor task launch worker for task 142 INFO Executor: Running task 2.0 in stage 44.0 (TID 142) 20/08/03 12:50:44.517 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 44.0 (TID 143, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.517 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 44.0 (TID 140) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:44.517 Executor task launch worker for task 143 INFO Executor: Running task 3.0 in stage 44.0 (TID 143) 20/08/03 12:50:44.517 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 44.0 (TID 141) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:44.518 Executor task launch worker for task 142 INFO BlockManager: Found block rdd_193_2 locally 20/08/03 12:50:44.519 Executor task launch worker for task 143 INFO BlockManager: Found block rdd_193_3 locally 20/08/03 12:50:44.519 Executor task launch worker for task 142 INFO Executor: Finished task 2.0 in stage 44.0 (TID 142). 2296 bytes result sent to driver 20/08/03 12:50:44.519 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 44.0 (TID 142) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:44.519 Executor task launch worker for task 143 INFO Executor: Finished task 3.0 in stage 44.0 (TID 143). 2296 bytes result sent to driver 20/08/03 12:50:44.520 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 44.0 (TID 143) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:44.520 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 44.0, whose tasks have all completed, from pool 20/08/03 12:50:44.520 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 44 (treeAggregate at RDDLossFunction.scala:61) finished in 0.014 s 20/08/03 12:50:44.520 dag-scheduler-event-loop INFO DAGScheduler: Job 44 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:44.520 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 44: Stage finished 20/08/03 12:50:44.520 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 44 finished: treeAggregate at RDDLossFunction.scala:61, took 0.015996 s 20/08/03 12:50:44.521 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(86) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:44.522 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_86_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:44.523 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_88 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:44.524 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_88_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:44.524 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_88_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:44.525 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 88 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:44.532 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:44.532 dag-scheduler-event-loop INFO DAGScheduler: Got job 45 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:44.532 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 45 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:44.532 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:44.533 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:44.533 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 45 (MapPartitionsRDD[202] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:44.534 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_89 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:44.536 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_89_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:44.536 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_89_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:44.537 dag-scheduler-event-loop INFO SparkContext: Created broadcast 89 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:44.537 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 45 (MapPartitionsRDD[202] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:44.537 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 45.0 with 4 tasks 20/08/03 12:50:44.538 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 45.0 (TID 144, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.538 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 45.0 (TID 145, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.538 Executor task launch worker for task 144 INFO Executor: Running task 0.0 in stage 45.0 (TID 144) 20/08/03 12:50:44.538 Executor task launch worker for task 145 INFO Executor: Running task 1.0 in stage 45.0 (TID 145) 20/08/03 12:50:44.542 Executor task launch worker for task 145 INFO BlockManager: Found block rdd_193_1 locally 20/08/03 12:50:44.542 Executor task launch worker for task 144 INFO BlockManager: Found block rdd_193_0 locally 20/08/03 12:50:44.543 Executor task launch worker for task 144 INFO Executor: Finished task 0.0 in stage 45.0 (TID 144). 2339 bytes result sent to driver 20/08/03 12:50:44.543 Executor task launch worker for task 145 INFO Executor: Finished task 1.0 in stage 45.0 (TID 145). 2339 bytes result sent to driver 20/08/03 12:50:44.543 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 45.0 (TID 146, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.544 Executor task launch worker for task 146 INFO Executor: Running task 2.0 in stage 45.0 (TID 146) 20/08/03 12:50:44.544 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 45.0 (TID 147, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.544 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 45.0 (TID 144) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:44.544 Executor task launch worker for task 147 INFO Executor: Running task 3.0 in stage 45.0 (TID 147) 20/08/03 12:50:44.544 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 45.0 (TID 145) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:44.547 Executor task launch worker for task 147 INFO BlockManager: Found block rdd_193_3 locally 20/08/03 12:50:44.547 Executor task launch worker for task 146 INFO BlockManager: Found block rdd_193_2 locally 20/08/03 12:50:44.547 Executor task launch worker for task 147 INFO Executor: Finished task 3.0 in stage 45.0 (TID 147). 2296 bytes result sent to driver 20/08/03 12:50:44.548 Executor task launch worker for task 146 INFO Executor: Finished task 2.0 in stage 45.0 (TID 146). 2296 bytes result sent to driver 20/08/03 12:50:44.548 task-result-getter-2 INFO TaskSetManager: Finished task 3.0 in stage 45.0 (TID 147) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:44.548 task-result-getter-3 INFO TaskSetManager: Finished task 2.0 in stage 45.0 (TID 146) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:44.548 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 45.0, whose tasks have all completed, from pool 20/08/03 12:50:44.549 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 45 (treeAggregate at RDDLossFunction.scala:61) finished in 0.016 s 20/08/03 12:50:44.549 dag-scheduler-event-loop INFO DAGScheduler: Job 45 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:44.549 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 45: Stage finished 20/08/03 12:50:44.549 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 45 finished: treeAggregate at RDDLossFunction.scala:61, took 0.017211 s 20/08/03 12:50:44.550 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(88) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:44.550 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO StrongWolfeLineSearch: Line search t: 0.5508985649472953 fval: 0.7342215028617264 rhs: 0.7358720183663033 cdd: 0.024057870964535308 20/08/03 12:50:44.550 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 0.5509 20/08/03 12:50:44.551 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_88_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:44.551 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.734222 (rel: 0.00225) 0.789814 20/08/03 12:50:44.551 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Converged because max iterations reached 20/08/03 12:50:44.551 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(83) (from destroy at LogisticRegression.scala:804) 20/08/03 12:50:44.551 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_83_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 85.0 B, free: 2.1 GiB) 20/08/03 12:50:44.554 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MapPartitionsRDD: Removing RDD 193 from persistence list 20/08/03 12:50:44.554 block-manager-slave-async-thread-pool-46 INFO BlockManager: Removing RDD 193 20/08/03 12:50:44.625 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [af4636b4] training finished 20/08/03 12:50:44.679 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5bac4c03] Stage class: LogisticRegression 20/08/03 12:50:44.679 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5bac4c03] Stage uid: logreg_02444d550004 20/08/03 12:50:44.707 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5bac4c03] training: numPartitions=4 storageLevel=StorageLevel(1 replicas) 20/08/03 12:50:44.707 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5bac4c03] {"predictionCol":"","probabilityCol":"probability","maxIter":1} 20/08/03 12:50:44.715 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at LogisticRegression.scala:499 20/08/03 12:50:44.716 dag-scheduler-event-loop INFO DAGScheduler: Got job 46 (treeAggregate at LogisticRegression.scala:499) with 4 output partitions 20/08/03 12:50:44.716 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 46 (treeAggregate at LogisticRegression.scala:499) 20/08/03 12:50:44.716 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:44.717 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:44.717 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 46 (MapPartitionsRDD[220] at treeAggregate at LogisticRegression.scala:499), which has no missing parents 20/08/03 12:50:44.724 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_90 stored as values in memory (estimated size 27.5 KiB, free 2.1 GiB) 20/08/03 12:50:44.726 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_90_piece0 stored as bytes in memory (estimated size 10.7 KiB, free 2.1 GiB) 20/08/03 12:50:44.726 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_90_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.7 KiB, free: 2.1 GiB) 20/08/03 12:50:44.726 dag-scheduler-event-loop INFO SparkContext: Created broadcast 90 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:44.727 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 46 (MapPartitionsRDD[220] at treeAggregate at LogisticRegression.scala:499) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:44.727 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 46.0 with 4 tasks 20/08/03 12:50:44.728 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 46.0 (TID 148, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.728 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 46.0 (TID 149, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.728 Executor task launch worker for task 148 INFO Executor: Running task 0.0 in stage 46.0 (TID 148) 20/08/03 12:50:44.728 Executor task launch worker for task 149 INFO Executor: Running task 1.0 in stage 46.0 (TID 149) 20/08/03 12:50:44.735 Executor task launch worker for task 148 INFO MemoryStore: Block rdd_214_0 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:44.735 Executor task launch worker for task 149 INFO MemoryStore: Block rdd_214_1 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:44.735 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_214_0 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:44.735 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_214_1 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:44.737 Executor task launch worker for task 148 INFO Executor: Finished task 0.0 in stage 46.0 (TID 148). 2673 bytes result sent to driver 20/08/03 12:50:44.737 Executor task launch worker for task 149 INFO Executor: Finished task 1.0 in stage 46.0 (TID 149). 2639 bytes result sent to driver 20/08/03 12:50:44.738 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 46.0 (TID 150, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.738 Executor task launch worker for task 150 INFO Executor: Running task 2.0 in stage 46.0 (TID 150) 20/08/03 12:50:44.738 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 46.0 (TID 151, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.738 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 46.0 (TID 148) in 10 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:44.738 Executor task launch worker for task 151 INFO Executor: Running task 3.0 in stage 46.0 (TID 151) 20/08/03 12:50:44.738 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 46.0 (TID 149) in 10 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:44.743 Executor task launch worker for task 150 INFO MemoryStore: Block rdd_214_2 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:44.743 Executor task launch worker for task 151 INFO MemoryStore: Block rdd_214_3 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:44.744 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_214_2 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:44.744 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_214_3 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:44.745 Executor task launch worker for task 150 INFO Executor: Finished task 2.0 in stage 46.0 (TID 150). 2673 bytes result sent to driver 20/08/03 12:50:44.745 Executor task launch worker for task 151 INFO Executor: Finished task 3.0 in stage 46.0 (TID 151). 2639 bytes result sent to driver 20/08/03 12:50:44.745 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 46.0 (TID 150) in 8 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:44.746 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 46.0 (TID 151) in 8 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:44.746 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 46.0, whose tasks have all completed, from pool 20/08/03 12:50:44.746 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 46 (treeAggregate at LogisticRegression.scala:499) finished in 0.029 s 20/08/03 12:50:44.746 dag-scheduler-event-loop INFO DAGScheduler: Job 46 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:44.746 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 46: Stage finished 20/08/03 12:50:44.746 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 46 finished: treeAggregate at LogisticRegression.scala:499, took 0.031057 s 20/08/03 12:50:44.747 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5bac4c03] {"numExamples":100} 20/08/03 12:50:44.747 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5bac4c03] {"lowestLabelWeight":"3.0"} 20/08/03 12:50:44.747 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5bac4c03] {"highestLabelWeight":"67.0"} 20/08/03 12:50:44.747 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5bac4c03] {"sumOfWeights":100.0} 20/08/03 12:50:44.748 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5bac4c03] {"numClasses":3} 20/08/03 12:50:44.748 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5bac4c03] {"numFeatures":2} 20/08/03 12:50:44.748 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_91 stored as values in memory (estimated size 56.0 B, free 2.1 GiB) 20/08/03 12:50:44.749 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_91_piece0 stored as bytes in memory (estimated size 85.0 B, free 2.1 GiB) 20/08/03 12:50:44.750 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_91_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 85.0 B, free: 2.1 GiB) 20/08/03 12:50:44.750 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 91 from broadcast at LogisticRegression.scala:593 20/08/03 12:50:44.751 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_92 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:44.752 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_92_piece0 stored as bytes in memory (estimated size 171.0 B, free 2.1 GiB) 20/08/03 12:50:44.752 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_92_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 171.0 B, free: 2.1 GiB) 20/08/03 12:50:44.752 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 92 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:44.759 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:44.760 dag-scheduler-event-loop INFO DAGScheduler: Got job 47 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:44.760 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 47 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:44.760 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:44.760 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:44.761 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 47 (MapPartitionsRDD[221] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:44.763 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_93 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:44.764 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_93_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:44.765 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_93_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:44.765 dag-scheduler-event-loop INFO SparkContext: Created broadcast 93 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:44.765 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 47 (MapPartitionsRDD[221] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:44.765 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 47.0 with 4 tasks 20/08/03 12:50:44.766 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 47.0 (TID 152, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.767 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 47.0 (TID 153, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.767 Executor task launch worker for task 152 INFO Executor: Running task 0.0 in stage 47.0 (TID 152) 20/08/03 12:50:44.767 Executor task launch worker for task 153 INFO Executor: Running task 1.0 in stage 47.0 (TID 153) 20/08/03 12:50:44.770 Executor task launch worker for task 153 INFO BlockManager: Found block rdd_214_1 locally 20/08/03 12:50:44.770 Executor task launch worker for task 152 INFO BlockManager: Found block rdd_214_0 locally 20/08/03 12:50:44.771 Executor task launch worker for task 152 INFO Executor: Finished task 0.0 in stage 47.0 (TID 152). 2339 bytes result sent to driver 20/08/03 12:50:44.771 Executor task launch worker for task 153 INFO Executor: Finished task 1.0 in stage 47.0 (TID 153). 2339 bytes result sent to driver 20/08/03 12:50:44.771 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 47.0 (TID 154, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.771 Executor task launch worker for task 154 INFO Executor: Running task 2.0 in stage 47.0 (TID 154) 20/08/03 12:50:44.771 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 47.0 (TID 155, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.771 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 47.0 (TID 152) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:44.771 Executor task launch worker for task 155 INFO Executor: Running task 3.0 in stage 47.0 (TID 155) 20/08/03 12:50:44.772 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 47.0 (TID 153) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:44.773 Executor task launch worker for task 155 INFO BlockManager: Found block rdd_214_3 locally 20/08/03 12:50:44.774 Executor task launch worker for task 155 INFO Executor: Finished task 3.0 in stage 47.0 (TID 155). 2296 bytes result sent to driver 20/08/03 12:50:44.774 Executor task launch worker for task 154 INFO BlockManager: Found block rdd_214_2 locally 20/08/03 12:50:44.774 task-result-getter-2 INFO TaskSetManager: Finished task 3.0 in stage 47.0 (TID 155) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:44.774 Executor task launch worker for task 154 INFO Executor: Finished task 2.0 in stage 47.0 (TID 154). 2296 bytes result sent to driver 20/08/03 12:50:44.774 task-result-getter-3 INFO TaskSetManager: Finished task 2.0 in stage 47.0 (TID 154) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:44.775 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 47.0, whose tasks have all completed, from pool 20/08/03 12:50:44.775 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 47 (treeAggregate at RDDLossFunction.scala:61) finished in 0.014 s 20/08/03 12:50:44.775 dag-scheduler-event-loop INFO DAGScheduler: Job 47 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:44.775 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 47: Stage finished 20/08/03 12:50:44.775 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 47 finished: treeAggregate at RDDLossFunction.scala:61, took 0.015900 s 20/08/03 12:50:44.776 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(92) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:44.777 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_92_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 171.0 B, free: 2.1 GiB) 20/08/03 12:50:44.777 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_94 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:44.778 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_94_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:44.778 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_94_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:44.779 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 94 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:44.785 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:44.786 dag-scheduler-event-loop INFO DAGScheduler: Got job 48 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:44.786 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 48 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:44.786 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:44.786 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:44.786 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 48 (MapPartitionsRDD[222] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:44.788 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_95 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:44.790 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_95_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:44.790 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_95_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:44.790 dag-scheduler-event-loop INFO SparkContext: Created broadcast 95 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:44.791 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 48 (MapPartitionsRDD[222] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:44.791 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 48.0 with 4 tasks 20/08/03 12:50:44.792 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 48.0 (TID 156, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.792 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 48.0 (TID 157, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.793 Executor task launch worker for task 156 INFO Executor: Running task 0.0 in stage 48.0 (TID 156) 20/08/03 12:50:44.793 Executor task launch worker for task 157 INFO Executor: Running task 1.0 in stage 48.0 (TID 157) 20/08/03 12:50:44.795 Executor task launch worker for task 157 INFO BlockManager: Found block rdd_214_1 locally 20/08/03 12:50:44.795 Executor task launch worker for task 156 INFO BlockManager: Found block rdd_214_0 locally 20/08/03 12:50:44.796 Executor task launch worker for task 157 INFO Executor: Finished task 1.0 in stage 48.0 (TID 157). 2296 bytes result sent to driver 20/08/03 12:50:44.796 Executor task launch worker for task 156 INFO Executor: Finished task 0.0 in stage 48.0 (TID 156). 2296 bytes result sent to driver 20/08/03 12:50:44.796 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 48.0 (TID 158, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.796 Executor task launch worker for task 158 INFO Executor: Running task 2.0 in stage 48.0 (TID 158) 20/08/03 12:50:44.797 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 48.0 (TID 159, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.797 Executor task launch worker for task 159 INFO Executor: Running task 3.0 in stage 48.0 (TID 159) 20/08/03 12:50:44.797 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 48.0 (TID 157) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:44.797 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 48.0 (TID 156) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:44.798 Executor task launch worker for task 158 INFO BlockManager: Found block rdd_214_2 locally 20/08/03 12:50:44.799 Executor task launch worker for task 158 INFO Executor: Finished task 2.0 in stage 48.0 (TID 158). 2296 bytes result sent to driver 20/08/03 12:50:44.799 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 48.0 (TID 158) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:44.800 Executor task launch worker for task 159 INFO BlockManager: Found block rdd_214_3 locally 20/08/03 12:50:44.800 Executor task launch worker for task 159 INFO Executor: Finished task 3.0 in stage 48.0 (TID 159). 2296 bytes result sent to driver 20/08/03 12:50:44.801 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 48.0 (TID 159) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:44.801 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 48.0, whose tasks have all completed, from pool 20/08/03 12:50:44.802 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 48 (treeAggregate at RDDLossFunction.scala:61) finished in 0.014 s 20/08/03 12:50:44.802 dag-scheduler-event-loop INFO DAGScheduler: Job 48 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:44.802 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 48: Stage finished 20/08/03 12:50:44.802 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 48 finished: treeAggregate at RDDLossFunction.scala:61, took 0.016897 s 20/08/03 12:50:44.803 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(94) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:44.804 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_94_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:44.804 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_96 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:44.805 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_96_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:44.805 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_96_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:44.806 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 96 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:44.816 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:44.816 dag-scheduler-event-loop INFO DAGScheduler: Got job 49 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:44.816 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 49 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:44.816 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:44.817 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:44.817 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 49 (MapPartitionsRDD[223] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:44.819 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_97 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:44.820 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_97_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:44.820 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_97_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:44.820 dag-scheduler-event-loop INFO SparkContext: Created broadcast 97 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:44.821 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 49 (MapPartitionsRDD[223] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:44.821 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 49.0 with 4 tasks 20/08/03 12:50:44.821 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 49.0 (TID 160, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.822 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 49.0 (TID 161, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.822 Executor task launch worker for task 160 INFO Executor: Running task 0.0 in stage 49.0 (TID 160) 20/08/03 12:50:44.822 Executor task launch worker for task 161 INFO Executor: Running task 1.0 in stage 49.0 (TID 161) 20/08/03 12:50:44.824 Executor task launch worker for task 160 INFO BlockManager: Found block rdd_214_0 locally 20/08/03 12:50:44.824 Executor task launch worker for task 161 INFO BlockManager: Found block rdd_214_1 locally 20/08/03 12:50:44.825 Executor task launch worker for task 160 INFO Executor: Finished task 0.0 in stage 49.0 (TID 160). 2296 bytes result sent to driver 20/08/03 12:50:44.825 Executor task launch worker for task 161 INFO Executor: Finished task 1.0 in stage 49.0 (TID 161). 2296 bytes result sent to driver 20/08/03 12:50:44.825 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 49.0 (TID 162, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.825 Executor task launch worker for task 162 INFO Executor: Running task 2.0 in stage 49.0 (TID 162) 20/08/03 12:50:44.825 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 49.0 (TID 163, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.826 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 49.0 (TID 160) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:44.826 Executor task launch worker for task 163 INFO Executor: Running task 3.0 in stage 49.0 (TID 163) 20/08/03 12:50:44.826 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 49.0 (TID 161) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:44.827 Executor task launch worker for task 162 INFO BlockManager: Found block rdd_214_2 locally 20/08/03 12:50:44.827 Executor task launch worker for task 162 INFO Executor: Finished task 2.0 in stage 49.0 (TID 162). 2296 bytes result sent to driver 20/08/03 12:50:44.828 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 49.0 (TID 162) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:44.828 Executor task launch worker for task 163 INFO BlockManager: Found block rdd_214_3 locally 20/08/03 12:50:44.828 Executor task launch worker for task 163 INFO Executor: Finished task 3.0 in stage 49.0 (TID 163). 2296 bytes result sent to driver 20/08/03 12:50:44.829 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 49.0 (TID 163) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:44.829 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 49.0, whose tasks have all completed, from pool 20/08/03 12:50:44.829 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 49 (treeAggregate at RDDLossFunction.scala:61) finished in 0.011 s 20/08/03 12:50:44.829 dag-scheduler-event-loop INFO DAGScheduler: Job 49 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:44.829 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 49: Stage finished 20/08/03 12:50:44.829 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 49 finished: treeAggregate at RDDLossFunction.scala:61, took 0.013504 s 20/08/03 12:50:44.830 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(96) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:44.831 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO StrongWolfeLineSearch: Line search t: 0.5508985649472953 fval: 0.7342215028617264 rhs: 0.7358720183663033 cdd: 0.024057870964535294 20/08/03 12:50:44.831 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 0.5509 20/08/03 12:50:44.831 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_96_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:44.831 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.734222 (rel: 0.00225) 0.789814 20/08/03 12:50:44.831 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Converged because max iterations reached 20/08/03 12:50:44.831 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(91) (from destroy at LogisticRegression.scala:804) 20/08/03 12:50:44.832 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MapPartitionsRDD: Removing RDD 214 from persistence list 20/08/03 12:50:44.832 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_91_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 85.0 B, free: 2.1 GiB) 20/08/03 12:50:44.832 block-manager-slave-async-thread-pool-61 INFO BlockManager: Removing RDD 214 20/08/03 12:50:44.900 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [5bac4c03] training finished 20/08/03 12:50:44.946 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [68f13dde] Stage class: LogisticRegression 20/08/03 12:50:44.946 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [68f13dde] Stage uid: logreg_02444d550004 20/08/03 12:50:44.974 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [68f13dde] training: numPartitions=4 storageLevel=StorageLevel(1 replicas) 20/08/03 12:50:44.975 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [68f13dde] {"predictionCol":"","probabilityCol":"","maxIter":1} 20/08/03 12:50:44.983 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at LogisticRegression.scala:499 20/08/03 12:50:44.983 dag-scheduler-event-loop INFO DAGScheduler: Got job 50 (treeAggregate at LogisticRegression.scala:499) with 4 output partitions 20/08/03 12:50:44.984 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 50 (treeAggregate at LogisticRegression.scala:499) 20/08/03 12:50:44.984 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:44.984 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:44.984 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 50 (MapPartitionsRDD[241] at treeAggregate at LogisticRegression.scala:499), which has no missing parents 20/08/03 12:50:44.992 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_98 stored as values in memory (estimated size 27.5 KiB, free 2.1 GiB) 20/08/03 12:50:44.994 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_98_piece0 stored as bytes in memory (estimated size 10.7 KiB, free 2.1 GiB) 20/08/03 12:50:44.994 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_98_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.7 KiB, free: 2.1 GiB) 20/08/03 12:50:44.994 dag-scheduler-event-loop INFO SparkContext: Created broadcast 98 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:44.994 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 50 (MapPartitionsRDD[241] at treeAggregate at LogisticRegression.scala:499) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:44.994 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 50.0 with 4 tasks 20/08/03 12:50:44.996 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 50.0 (TID 164, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.996 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 50.0 (TID 165, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:44.996 Executor task launch worker for task 164 INFO Executor: Running task 0.0 in stage 50.0 (TID 164) 20/08/03 12:50:44.996 Executor task launch worker for task 165 INFO Executor: Running task 1.0 in stage 50.0 (TID 165) 20/08/03 12:50:45.002 Executor task launch worker for task 165 INFO MemoryStore: Block rdd_235_1 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:45.003 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_235_1 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:45.004 Executor task launch worker for task 165 INFO Executor: Finished task 1.0 in stage 50.0 (TID 165). 2639 bytes result sent to driver 20/08/03 12:50:45.005 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 50.0 (TID 166, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.005 Executor task launch worker for task 166 INFO Executor: Running task 2.0 in stage 50.0 (TID 166) 20/08/03 12:50:45.005 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 50.0 (TID 165) in 9 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:45.006 Executor task launch worker for task 164 INFO MemoryStore: Block rdd_235_0 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:45.007 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_235_0 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:45.008 Executor task launch worker for task 164 INFO Executor: Finished task 0.0 in stage 50.0 (TID 164). 2673 bytes result sent to driver 20/08/03 12:50:45.009 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 50.0 (TID 167, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.009 Executor task launch worker for task 167 INFO Executor: Running task 3.0 in stage 50.0 (TID 167) 20/08/03 12:50:45.010 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 50.0 (TID 164) in 15 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:45.011 Executor task launch worker for task 166 INFO MemoryStore: Block rdd_235_2 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:45.012 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_235_2 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:45.012 Executor task launch worker for task 166 INFO Executor: Finished task 2.0 in stage 50.0 (TID 166). 2673 bytes result sent to driver 20/08/03 12:50:45.013 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 50.0 (TID 166) in 9 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:45.017 Executor task launch worker for task 167 INFO MemoryStore: Block rdd_235_3 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:45.017 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_235_3 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:45.017 Executor task launch worker for task 167 INFO Executor: Finished task 3.0 in stage 50.0 (TID 167). 2639 bytes result sent to driver 20/08/03 12:50:45.018 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 50.0 (TID 167) in 9 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:45.018 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 50.0, whose tasks have all completed, from pool 20/08/03 12:50:45.019 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 50 (treeAggregate at LogisticRegression.scala:499) finished in 0.034 s 20/08/03 12:50:45.019 dag-scheduler-event-loop INFO DAGScheduler: Job 50 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:45.019 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 50: Stage finished 20/08/03 12:50:45.019 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 50 finished: treeAggregate at LogisticRegression.scala:499, took 0.036531 s 20/08/03 12:50:45.020 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [68f13dde] {"numExamples":100} 20/08/03 12:50:45.020 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [68f13dde] {"lowestLabelWeight":"3.0"} 20/08/03 12:50:45.021 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [68f13dde] {"highestLabelWeight":"67.0"} 20/08/03 12:50:45.021 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [68f13dde] {"sumOfWeights":100.0} 20/08/03 12:50:45.021 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [68f13dde] {"numClasses":3} 20/08/03 12:50:45.021 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [68f13dde] {"numFeatures":2} 20/08/03 12:50:45.022 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_99 stored as values in memory (estimated size 56.0 B, free 2.1 GiB) 20/08/03 12:50:45.024 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_99_piece0 stored as bytes in memory (estimated size 85.0 B, free 2.1 GiB) 20/08/03 12:50:45.025 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_99_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 85.0 B, free: 2.1 GiB) 20/08/03 12:50:45.025 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 99 from broadcast at LogisticRegression.scala:593 20/08/03 12:50:45.026 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_100 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:45.027 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_100_piece0 stored as bytes in memory (estimated size 171.0 B, free 2.1 GiB) 20/08/03 12:50:45.027 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_100_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 171.0 B, free: 2.1 GiB) 20/08/03 12:50:45.028 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 100 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:45.034 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:45.151 dag-scheduler-event-loop INFO DAGScheduler: Got job 51 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:45.151 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 51 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:45.151 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:45.152 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:45.152 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 51 (MapPartitionsRDD[242] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:45.155 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_101 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:45.157 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_101_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:45.157 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_101_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.158 dag-scheduler-event-loop INFO SparkContext: Created broadcast 101 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:45.158 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 51 (MapPartitionsRDD[242] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:45.158 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 51.0 with 4 tasks 20/08/03 12:50:45.159 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 51.0 (TID 168, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.159 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 51.0 (TID 169, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.159 Executor task launch worker for task 168 INFO Executor: Running task 0.0 in stage 51.0 (TID 168) 20/08/03 12:50:45.159 Executor task launch worker for task 169 INFO Executor: Running task 1.0 in stage 51.0 (TID 169) 20/08/03 12:50:45.162 Executor task launch worker for task 168 INFO BlockManager: Found block rdd_235_0 locally 20/08/03 12:50:45.162 Executor task launch worker for task 169 INFO BlockManager: Found block rdd_235_1 locally 20/08/03 12:50:45.163 Executor task launch worker for task 168 INFO Executor: Finished task 0.0 in stage 51.0 (TID 168). 2296 bytes result sent to driver 20/08/03 12:50:45.163 Executor task launch worker for task 169 INFO Executor: Finished task 1.0 in stage 51.0 (TID 169). 2296 bytes result sent to driver 20/08/03 12:50:45.163 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 51.0 (TID 170, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.163 Executor task launch worker for task 170 INFO Executor: Running task 2.0 in stage 51.0 (TID 170) 20/08/03 12:50:45.163 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 51.0 (TID 171, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.163 Executor task launch worker for task 171 INFO Executor: Running task 3.0 in stage 51.0 (TID 171) 20/08/03 12:50:45.163 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 51.0 (TID 168) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:45.164 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 51.0 (TID 169) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:45.165 Executor task launch worker for task 171 INFO BlockManager: Found block rdd_235_3 locally 20/08/03 12:50:45.165 Executor task launch worker for task 170 INFO BlockManager: Found block rdd_235_2 locally 20/08/03 12:50:45.166 Executor task launch worker for task 171 INFO Executor: Finished task 3.0 in stage 51.0 (TID 171). 2296 bytes result sent to driver 20/08/03 12:50:45.166 Executor task launch worker for task 170 INFO Executor: Finished task 2.0 in stage 51.0 (TID 170). 2296 bytes result sent to driver 20/08/03 12:50:45.166 task-result-getter-2 INFO TaskSetManager: Finished task 3.0 in stage 51.0 (TID 171) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:45.166 task-result-getter-3 INFO TaskSetManager: Finished task 2.0 in stage 51.0 (TID 170) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:45.167 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 51.0, whose tasks have all completed, from pool 20/08/03 12:50:45.167 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 51 (treeAggregate at RDDLossFunction.scala:61) finished in 0.014 s 20/08/03 12:50:45.167 dag-scheduler-event-loop INFO DAGScheduler: Job 51 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:45.167 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 51: Stage finished 20/08/03 12:50:45.167 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 51 finished: treeAggregate at RDDLossFunction.scala:61, took 0.017354 s 20/08/03 12:50:45.168 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(100) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:45.169 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_100_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 171.0 B, free: 2.1 GiB) 20/08/03 12:50:45.169 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_102 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:45.170 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_102_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:45.171 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_102_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:45.171 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 102 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:45.179 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:45.180 dag-scheduler-event-loop INFO DAGScheduler: Got job 52 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:45.180 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 52 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:45.180 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:45.180 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:45.181 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 52 (MapPartitionsRDD[243] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:45.182 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_103 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:45.184 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_103_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:45.184 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_103_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.184 dag-scheduler-event-loop INFO SparkContext: Created broadcast 103 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:45.185 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 52 (MapPartitionsRDD[243] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:45.185 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 52.0 with 4 tasks 20/08/03 12:50:45.186 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 52.0 (TID 172, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.186 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 52.0 (TID 173, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.186 Executor task launch worker for task 173 INFO Executor: Running task 1.0 in stage 52.0 (TID 173) 20/08/03 12:50:45.186 Executor task launch worker for task 172 INFO Executor: Running task 0.0 in stage 52.0 (TID 172) 20/08/03 12:50:45.188 Executor task launch worker for task 173 INFO BlockManager: Found block rdd_235_1 locally 20/08/03 12:50:45.189 Executor task launch worker for task 173 INFO Executor: Finished task 1.0 in stage 52.0 (TID 173). 2296 bytes result sent to driver 20/08/03 12:50:45.189 Executor task launch worker for task 172 INFO BlockManager: Found block rdd_235_0 locally 20/08/03 12:50:45.189 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 52.0 (TID 174, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.190 Executor task launch worker for task 174 INFO Executor: Running task 2.0 in stage 52.0 (TID 174) 20/08/03 12:50:45.190 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 52.0 (TID 173) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:45.190 Executor task launch worker for task 172 INFO Executor: Finished task 0.0 in stage 52.0 (TID 172). 2296 bytes result sent to driver 20/08/03 12:50:45.191 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 52.0 (TID 175, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.191 Executor task launch worker for task 175 INFO Executor: Running task 3.0 in stage 52.0 (TID 175) 20/08/03 12:50:45.191 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 52.0 (TID 172) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:45.191 Executor task launch worker for task 174 INFO BlockManager: Found block rdd_235_2 locally 20/08/03 12:50:45.192 Executor task launch worker for task 174 INFO Executor: Finished task 2.0 in stage 52.0 (TID 174). 2296 bytes result sent to driver 20/08/03 12:50:45.192 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 52.0 (TID 174) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:45.194 Executor task launch worker for task 175 INFO BlockManager: Found block rdd_235_3 locally 20/08/03 12:50:45.195 Executor task launch worker for task 175 INFO Executor: Finished task 3.0 in stage 52.0 (TID 175). 2296 bytes result sent to driver 20/08/03 12:50:45.195 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 52.0 (TID 175) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:45.195 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 52.0, whose tasks have all completed, from pool 20/08/03 12:50:45.196 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 52 (treeAggregate at RDDLossFunction.scala:61) finished in 0.014 s 20/08/03 12:50:45.196 dag-scheduler-event-loop INFO DAGScheduler: Job 52 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:45.196 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 52: Stage finished 20/08/03 12:50:45.196 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 52 finished: treeAggregate at RDDLossFunction.scala:61, took 0.016565 s 20/08/03 12:50:45.196 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(102) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:45.197 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_102_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:45.198 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_104 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:45.199 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_104_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:45.199 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_104_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:45.199 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 104 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:45.206 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:45.206 dag-scheduler-event-loop INFO DAGScheduler: Got job 53 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:45.206 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 53 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:45.206 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:45.206 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:45.207 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 53 (MapPartitionsRDD[244] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:45.208 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_105 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:45.210 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_105_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:45.210 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_105_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.210 dag-scheduler-event-loop INFO SparkContext: Created broadcast 105 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:45.210 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 53 (MapPartitionsRDD[244] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:45.210 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 53.0 with 4 tasks 20/08/03 12:50:45.211 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 53.0 (TID 176, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.211 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 53.0 (TID 177, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.212 Executor task launch worker for task 177 INFO Executor: Running task 1.0 in stage 53.0 (TID 177) 20/08/03 12:50:45.212 Executor task launch worker for task 176 INFO Executor: Running task 0.0 in stage 53.0 (TID 176) 20/08/03 12:50:45.214 Executor task launch worker for task 177 INFO BlockManager: Found block rdd_235_1 locally 20/08/03 12:50:45.214 Executor task launch worker for task 176 INFO BlockManager: Found block rdd_235_0 locally 20/08/03 12:50:45.214 Executor task launch worker for task 177 INFO Executor: Finished task 1.0 in stage 53.0 (TID 177). 2296 bytes result sent to driver 20/08/03 12:50:45.215 Executor task launch worker for task 176 INFO Executor: Finished task 0.0 in stage 53.0 (TID 176). 2296 bytes result sent to driver 20/08/03 12:50:45.215 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 53.0 (TID 178, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.215 Executor task launch worker for task 178 INFO Executor: Running task 2.0 in stage 53.0 (TID 178) 20/08/03 12:50:45.215 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 53.0 (TID 179, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.215 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 53.0 (TID 177) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:45.215 Executor task launch worker for task 179 INFO Executor: Running task 3.0 in stage 53.0 (TID 179) 20/08/03 12:50:45.215 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 53.0 (TID 176) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:45.217 Executor task launch worker for task 179 INFO BlockManager: Found block rdd_235_3 locally 20/08/03 12:50:45.217 Executor task launch worker for task 178 INFO BlockManager: Found block rdd_235_2 locally 20/08/03 12:50:45.217 Executor task launch worker for task 179 INFO Executor: Finished task 3.0 in stage 53.0 (TID 179). 2296 bytes result sent to driver 20/08/03 12:50:45.218 Executor task launch worker for task 178 INFO Executor: Finished task 2.0 in stage 53.0 (TID 178). 2296 bytes result sent to driver 20/08/03 12:50:45.218 task-result-getter-2 INFO TaskSetManager: Finished task 3.0 in stage 53.0 (TID 179) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:45.218 task-result-getter-3 INFO TaskSetManager: Finished task 2.0 in stage 53.0 (TID 178) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:45.218 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 53.0, whose tasks have all completed, from pool 20/08/03 12:50:45.218 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 53 (treeAggregate at RDDLossFunction.scala:61) finished in 0.011 s 20/08/03 12:50:45.219 dag-scheduler-event-loop INFO DAGScheduler: Job 53 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:45.219 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 53: Stage finished 20/08/03 12:50:45.219 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 53 finished: treeAggregate at RDDLossFunction.scala:61, took 0.013252 s 20/08/03 12:50:45.219 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(104) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:45.220 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO StrongWolfeLineSearch: Line search t: 0.5508985649472953 fval: 0.7342215028617264 rhs: 0.7358720183663033 cdd: 0.024057870964535308 20/08/03 12:50:45.220 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 0.5509 20/08/03 12:50:45.220 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.734222 (rel: 0.00225) 0.789814 20/08/03 12:50:45.220 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_104_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:45.220 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Converged because max iterations reached 20/08/03 12:50:45.220 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(99) (from destroy at LogisticRegression.scala:804) 20/08/03 12:50:45.221 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MapPartitionsRDD: Removing RDD 235 from persistence list 20/08/03 12:50:45.221 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_99_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 85.0 B, free: 2.1 GiB) 20/08/03 12:50:45.221 block-manager-slave-async-thread-pool-74 INFO BlockManager: Removing RDD 235 20/08/03 12:50:45.288 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [68f13dde] training finished 20/08/03 12:50:45.288 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: ===== FINISHED o.a.s.ml.classification.LogisticRegressionSuite: 'empty probabilityCol or predictionCol' ===== 20/08/03 12:50:45.290 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: ===== TEST OUTPUT FOR o.a.s.ml.classification.LogisticRegressionSuite: 'check summary types for binary and multiclass' ===== 20/08/03 12:50:45.322 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [fa406561] Stage class: LogisticRegression 20/08/03 12:50:45.322 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [fa406561] Stage uid: logreg_2eb1c67f8c01 20/08/03 12:50:45.343 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [fa406561] training: numPartitions=2 storageLevel=StorageLevel(1 replicas) 20/08/03 12:50:45.344 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [fa406561] {"maxIter":1} 20/08/03 12:50:45.351 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at LogisticRegression.scala:499 20/08/03 12:50:45.352 dag-scheduler-event-loop INFO DAGScheduler: Got job 54 (treeAggregate at LogisticRegression.scala:499) with 2 output partitions 20/08/03 12:50:45.352 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 54 (treeAggregate at LogisticRegression.scala:499) 20/08/03 12:50:45.352 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:45.352 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:45.353 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 54 (MapPartitionsRDD[262] at treeAggregate at LogisticRegression.scala:499), which has no missing parents 20/08/03 12:50:45.357 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_106 stored as values in memory (estimated size 12.6 KiB, free 2.1 GiB) 20/08/03 12:50:45.358 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_106_piece0 stored as bytes in memory (estimated size 6.4 KiB, free 2.1 GiB) 20/08/03 12:50:45.358 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_106_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 6.4 KiB, free: 2.1 GiB) 20/08/03 12:50:45.358 dag-scheduler-event-loop INFO SparkContext: Created broadcast 106 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:45.359 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 54 (MapPartitionsRDD[262] at treeAggregate at LogisticRegression.scala:499) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:45.359 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 54.0 with 2 tasks 20/08/03 12:50:45.359 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 54.0 (TID 180, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:45.359 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 54.0 (TID 181, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:45.360 Executor task launch worker for task 180 INFO Executor: Running task 0.0 in stage 54.0 (TID 180) 20/08/03 12:50:45.360 Executor task launch worker for task 181 INFO Executor: Running task 1.0 in stage 54.0 (TID 181) 20/08/03 12:50:45.365 Executor task launch worker for task 180 INFO MemoryStore: Block rdd_256_0 stored as values in memory (estimated size 3.7 KiB, free 2.1 GiB) 20/08/03 12:50:45.365 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_256_0 in memory on amp-jenkins-worker-06.amp:34426 (size: 3.7 KiB, free: 2.1 GiB) 20/08/03 12:50:45.365 Executor task launch worker for task 181 INFO MemoryStore: Block rdd_256_1 stored as values in memory (estimated size 3.7 KiB, free 2.1 GiB) 20/08/03 12:50:45.366 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_256_1 in memory on amp-jenkins-worker-06.amp:34426 (size: 3.7 KiB, free: 2.1 GiB) 20/08/03 12:50:45.366 Executor task launch worker for task 180 INFO Executor: Finished task 0.0 in stage 54.0 (TID 180). 2543 bytes result sent to driver 20/08/03 12:50:45.366 Executor task launch worker for task 181 INFO Executor: Finished task 1.0 in stage 54.0 (TID 181). 2543 bytes result sent to driver 20/08/03 12:50:45.367 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 54.0 (TID 180) in 8 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:45.367 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 54.0 (TID 181) in 8 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:45.367 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 54.0, whose tasks have all completed, from pool 20/08/03 12:50:45.368 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 54 (treeAggregate at LogisticRegression.scala:499) finished in 0.014 s 20/08/03 12:50:45.368 dag-scheduler-event-loop INFO DAGScheduler: Job 54 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:45.368 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 54: Stage finished 20/08/03 12:50:45.368 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 54 finished: treeAggregate at LogisticRegression.scala:499, took 0.016632 s 20/08/03 12:50:45.368 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [fa406561] {"numExamples":100} 20/08/03 12:50:45.369 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [fa406561] {"lowestLabelWeight":"34.0"} 20/08/03 12:50:45.369 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [fa406561] {"highestLabelWeight":"66.0"} 20/08/03 12:50:45.369 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [fa406561] {"sumOfWeights":100.0} 20/08/03 12:50:45.369 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [fa406561] {"numClasses":2} 20/08/03 12:50:45.369 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [fa406561] {"numFeatures":1} 20/08/03 12:50:45.369 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_107 stored as values in memory (estimated size 48.0 B, free 2.1 GiB) 20/08/03 12:50:45.370 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_107_piece0 stored as bytes in memory (estimated size 77.0 B, free 2.1 GiB) 20/08/03 12:50:45.371 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_107_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 77.0 B, free: 2.1 GiB) 20/08/03 12:50:45.371 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 107 from broadcast at LogisticRegression.scala:593 20/08/03 12:50:45.371 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_108 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:45.372 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_108_piece0 stored as bytes in memory (estimated size 153.0 B, free 2.1 GiB) 20/08/03 12:50:45.372 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_108_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 153.0 B, free: 2.1 GiB) 20/08/03 12:50:45.373 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 108 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:45.379 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:45.380 dag-scheduler-event-loop INFO DAGScheduler: Got job 55 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:45.380 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 55 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:45.380 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:45.380 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:45.380 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 55 (MapPartitionsRDD[263] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:45.382 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_109 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:45.384 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_109_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:45.384 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_109_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:45.384 dag-scheduler-event-loop INFO SparkContext: Created broadcast 109 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:45.384 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 55 (MapPartitionsRDD[263] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:45.384 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 55.0 with 2 tasks 20/08/03 12:50:45.385 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 55.0 (TID 182, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:45.386 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 55.0 (TID 183, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:45.386 Executor task launch worker for task 183 INFO Executor: Running task 1.0 in stage 55.0 (TID 183) 20/08/03 12:50:45.386 Executor task launch worker for task 182 INFO Executor: Running task 0.0 in stage 55.0 (TID 182) 20/08/03 12:50:45.388 Executor task launch worker for task 183 INFO BlockManager: Found block rdd_256_1 locally 20/08/03 12:50:45.388 Executor task launch worker for task 182 INFO BlockManager: Found block rdd_256_0 locally 20/08/03 12:50:45.389 Executor task launch worker for task 182 INFO Executor: Finished task 0.0 in stage 55.0 (TID 182). 2168 bytes result sent to driver 20/08/03 12:50:45.389 Executor task launch worker for task 183 INFO Executor: Finished task 1.0 in stage 55.0 (TID 183). 2168 bytes result sent to driver 20/08/03 12:50:45.389 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 55.0 (TID 182) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:45.389 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 55.0 (TID 183) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:45.389 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 55.0, whose tasks have all completed, from pool 20/08/03 12:50:45.390 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 55 (treeAggregate at RDDLossFunction.scala:61) finished in 0.009 s 20/08/03 12:50:45.390 dag-scheduler-event-loop INFO DAGScheduler: Job 55 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:45.390 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 55: Stage finished 20/08/03 12:50:45.390 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 55 finished: treeAggregate at RDDLossFunction.scala:61, took 0.010816 s 20/08/03 12:50:45.391 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(108) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:45.391 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_108_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 153.0 B, free: 2.1 GiB) 20/08/03 12:50:45.392 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_110 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:45.393 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_110_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:45.393 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_110_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:45.394 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 110 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:45.401 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:45.401 dag-scheduler-event-loop INFO DAGScheduler: Got job 56 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:45.401 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 56 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:45.401 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:45.402 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:45.402 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 56 (MapPartitionsRDD[264] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:45.403 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_111 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:45.405 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_111_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:45.405 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_111_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:45.406 dag-scheduler-event-loop INFO SparkContext: Created broadcast 111 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:45.406 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 56 (MapPartitionsRDD[264] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:45.406 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 56.0 with 2 tasks 20/08/03 12:50:45.407 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 56.0 (TID 184, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:45.407 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 56.0 (TID 185, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:45.407 Executor task launch worker for task 184 INFO Executor: Running task 0.0 in stage 56.0 (TID 184) 20/08/03 12:50:45.407 Executor task launch worker for task 185 INFO Executor: Running task 1.0 in stage 56.0 (TID 185) 20/08/03 12:50:45.409 Executor task launch worker for task 185 INFO BlockManager: Found block rdd_256_1 locally 20/08/03 12:50:45.409 Executor task launch worker for task 184 INFO BlockManager: Found block rdd_256_0 locally 20/08/03 12:50:45.410 Executor task launch worker for task 185 INFO Executor: Finished task 1.0 in stage 56.0 (TID 185). 2168 bytes result sent to driver 20/08/03 12:50:45.410 Executor task launch worker for task 184 INFO Executor: Finished task 0.0 in stage 56.0 (TID 184). 2168 bytes result sent to driver 20/08/03 12:50:45.410 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 56.0 (TID 185) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:45.411 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 56.0 (TID 184) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:45.411 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 56.0, whose tasks have all completed, from pool 20/08/03 12:50:45.411 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 56 (treeAggregate at RDDLossFunction.scala:61) finished in 0.009 s 20/08/03 12:50:45.411 dag-scheduler-event-loop INFO DAGScheduler: Job 56 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:45.411 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 56: Stage finished 20/08/03 12:50:45.411 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 56 finished: treeAggregate at RDDLossFunction.scala:61, took 0.010565 s 20/08/03 12:50:45.412 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(110) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:45.413 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 3.941 20/08/03 12:50:45.413 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.493391 (rel: 0.230) 0.0621158 20/08/03 12:50:45.413 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_110_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:45.413 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Converged because max iterations reached 20/08/03 12:50:45.413 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(107) (from destroy at LogisticRegression.scala:804) 20/08/03 12:50:45.414 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MapPartitionsRDD: Removing RDD 256 from persistence list 20/08/03 12:50:45.414 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_107_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 77.0 B, free: 2.1 GiB) 20/08/03 12:50:45.414 block-manager-slave-async-thread-pool-88 INFO BlockManager: Removing RDD 256 20/08/03 12:50:45.494 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [fa406561] training finished 20/08/03 12:50:45.540 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [a96f2526] Stage class: LogisticRegression 20/08/03 12:50:45.540 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [a96f2526] Stage uid: logreg_2eb1c67f8c01 20/08/03 12:50:45.570 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [a96f2526] training: numPartitions=4 storageLevel=StorageLevel(1 replicas) 20/08/03 12:50:45.570 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [a96f2526] {"maxIter":1} 20/08/03 12:50:45.579 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at LogisticRegression.scala:499 20/08/03 12:50:45.580 dag-scheduler-event-loop INFO DAGScheduler: Got job 57 (treeAggregate at LogisticRegression.scala:499) with 4 output partitions 20/08/03 12:50:45.580 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 57 (treeAggregate at LogisticRegression.scala:499) 20/08/03 12:50:45.580 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:45.581 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:45.581 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 57 (MapPartitionsRDD[289] at treeAggregate at LogisticRegression.scala:499), which has no missing parents 20/08/03 12:50:45.587 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_112 stored as values in memory (estimated size 27.5 KiB, free 2.1 GiB) 20/08/03 12:50:45.590 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_112_piece0 stored as bytes in memory (estimated size 10.8 KiB, free 2.1 GiB) 20/08/03 12:50:45.590 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_112_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.8 KiB, free: 2.1 GiB) 20/08/03 12:50:45.590 dag-scheduler-event-loop INFO SparkContext: Created broadcast 112 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:45.591 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 57 (MapPartitionsRDD[289] at treeAggregate at LogisticRegression.scala:499) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:45.591 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 57.0 with 4 tasks 20/08/03 12:50:45.592 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 57.0 (TID 186, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.592 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 57.0 (TID 187, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.592 Executor task launch worker for task 186 INFO Executor: Running task 0.0 in stage 57.0 (TID 186) 20/08/03 12:50:45.592 Executor task launch worker for task 187 INFO Executor: Running task 1.0 in stage 57.0 (TID 187) 20/08/03 12:50:45.597 Executor task launch worker for task 187 INFO MemoryStore: Block rdd_283_1 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:45.597 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_283_1 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:45.598 Executor task launch worker for task 186 INFO MemoryStore: Block rdd_283_0 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:45.598 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_283_0 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:45.598 Executor task launch worker for task 187 INFO Executor: Finished task 1.0 in stage 57.0 (TID 187). 2639 bytes result sent to driver 20/08/03 12:50:45.599 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 57.0 (TID 188, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.599 Executor task launch worker for task 186 INFO Executor: Finished task 0.0 in stage 57.0 (TID 186). 2673 bytes result sent to driver 20/08/03 12:50:45.599 Executor task launch worker for task 188 INFO Executor: Running task 2.0 in stage 57.0 (TID 188) 20/08/03 12:50:45.599 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 57.0 (TID 189, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.600 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 57.0 (TID 187) in 8 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:45.600 Executor task launch worker for task 189 INFO Executor: Running task 3.0 in stage 57.0 (TID 189) 20/08/03 12:50:45.600 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 57.0 (TID 186) in 8 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:45.603 Executor task launch worker for task 188 INFO MemoryStore: Block rdd_283_2 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:45.603 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_283_2 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:45.604 Executor task launch worker for task 189 INFO MemoryStore: Block rdd_283_3 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:45.604 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_283_3 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:45.604 Executor task launch worker for task 188 INFO Executor: Finished task 2.0 in stage 57.0 (TID 188). 2673 bytes result sent to driver 20/08/03 12:50:45.605 Executor task launch worker for task 189 INFO Executor: Finished task 3.0 in stage 57.0 (TID 189). 2639 bytes result sent to driver 20/08/03 12:50:45.605 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 57.0 (TID 188) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:45.606 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 57.0 (TID 189) in 7 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:45.606 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 57.0, whose tasks have all completed, from pool 20/08/03 12:50:45.606 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 57 (treeAggregate at LogisticRegression.scala:499) finished in 0.024 s 20/08/03 12:50:45.606 dag-scheduler-event-loop INFO DAGScheduler: Job 57 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:45.606 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 57: Stage finished 20/08/03 12:50:45.607 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 57 finished: treeAggregate at LogisticRegression.scala:499, took 0.027445 s 20/08/03 12:50:45.607 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [a96f2526] {"numExamples":100} 20/08/03 12:50:45.607 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [a96f2526] {"lowestLabelWeight":"3.0"} 20/08/03 12:50:45.608 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [a96f2526] {"highestLabelWeight":"67.0"} 20/08/03 12:50:45.608 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [a96f2526] {"sumOfWeights":100.0} 20/08/03 12:50:45.608 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [a96f2526] {"numClasses":3} 20/08/03 12:50:45.608 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [a96f2526] {"numFeatures":2} 20/08/03 12:50:45.609 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_113 stored as values in memory (estimated size 56.0 B, free 2.1 GiB) 20/08/03 12:50:45.611 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_113_piece0 stored as bytes in memory (estimated size 85.0 B, free 2.1 GiB) 20/08/03 12:50:45.611 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_113_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 85.0 B, free: 2.1 GiB) 20/08/03 12:50:45.612 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 113 from broadcast at LogisticRegression.scala:593 20/08/03 12:50:45.613 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_114 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:45.616 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_114_piece0 stored as bytes in memory (estimated size 171.0 B, free 2.1 GiB) 20/08/03 12:50:45.616 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_114_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 171.0 B, free: 2.1 GiB) 20/08/03 12:50:45.616 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 114 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:45.623 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:45.624 dag-scheduler-event-loop INFO DAGScheduler: Got job 58 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:45.624 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 58 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:45.624 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:45.624 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:45.624 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 58 (MapPartitionsRDD[290] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:45.626 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_115 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:45.628 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_115_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:45.628 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_115_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.628 dag-scheduler-event-loop INFO SparkContext: Created broadcast 115 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:45.628 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 58 (MapPartitionsRDD[290] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:45.628 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 58.0 with 4 tasks 20/08/03 12:50:45.630 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 58.0 (TID 190, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.630 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 58.0 (TID 191, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.630 Executor task launch worker for task 190 INFO Executor: Running task 0.0 in stage 58.0 (TID 190) 20/08/03 12:50:45.630 Executor task launch worker for task 191 INFO Executor: Running task 1.0 in stage 58.0 (TID 191) 20/08/03 12:50:45.633 Executor task launch worker for task 190 INFO BlockManager: Found block rdd_283_0 locally 20/08/03 12:50:45.633 Executor task launch worker for task 191 INFO BlockManager: Found block rdd_283_1 locally 20/08/03 12:50:45.633 Executor task launch worker for task 190 INFO Executor: Finished task 0.0 in stage 58.0 (TID 190). 2296 bytes result sent to driver 20/08/03 12:50:45.633 Executor task launch worker for task 191 INFO Executor: Finished task 1.0 in stage 58.0 (TID 191). 2296 bytes result sent to driver 20/08/03 12:50:45.634 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 58.0 (TID 192, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.634 Executor task launch worker for task 192 INFO Executor: Running task 2.0 in stage 58.0 (TID 192) 20/08/03 12:50:45.634 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 58.0 (TID 193, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.635 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 58.0 (TID 190) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:45.635 Executor task launch worker for task 193 INFO Executor: Running task 3.0 in stage 58.0 (TID 193) 20/08/03 12:50:45.635 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 58.0 (TID 191) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:45.636 Executor task launch worker for task 192 INFO BlockManager: Found block rdd_283_2 locally 20/08/03 12:50:45.636 Executor task launch worker for task 193 INFO BlockManager: Found block rdd_283_3 locally 20/08/03 12:50:45.637 Executor task launch worker for task 193 INFO Executor: Finished task 3.0 in stage 58.0 (TID 193). 2296 bytes result sent to driver 20/08/03 12:50:45.637 Executor task launch worker for task 192 INFO Executor: Finished task 2.0 in stage 58.0 (TID 192). 2296 bytes result sent to driver 20/08/03 12:50:45.638 task-result-getter-0 INFO TaskSetManager: Finished task 3.0 in stage 58.0 (TID 193) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:45.638 task-result-getter-1 INFO TaskSetManager: Finished task 2.0 in stage 58.0 (TID 192) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:45.638 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 58.0, whose tasks have all completed, from pool 20/08/03 12:50:45.639 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 58 (treeAggregate at RDDLossFunction.scala:61) finished in 0.013 s 20/08/03 12:50:45.639 dag-scheduler-event-loop INFO DAGScheduler: Job 58 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:45.639 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 58: Stage finished 20/08/03 12:50:45.639 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 58 finished: treeAggregate at RDDLossFunction.scala:61, took 0.015580 s 20/08/03 12:50:45.639 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(114) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:45.640 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_114_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 171.0 B, free: 2.1 GiB) 20/08/03 12:50:45.641 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_116 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:45.643 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_116_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:45.643 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_116_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:45.644 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 116 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:45.651 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:45.651 dag-scheduler-event-loop INFO DAGScheduler: Got job 59 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:45.651 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 59 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:45.651 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:45.651 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:45.652 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 59 (MapPartitionsRDD[291] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:45.653 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_117 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:45.655 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_117_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:45.655 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_117_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.655 dag-scheduler-event-loop INFO SparkContext: Created broadcast 117 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:45.656 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 59 (MapPartitionsRDD[291] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:45.656 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 59.0 with 4 tasks 20/08/03 12:50:45.656 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 59.0 (TID 194, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.657 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 59.0 (TID 195, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.657 Executor task launch worker for task 195 INFO Executor: Running task 1.0 in stage 59.0 (TID 195) 20/08/03 12:50:45.657 Executor task launch worker for task 194 INFO Executor: Running task 0.0 in stage 59.0 (TID 194) 20/08/03 12:50:45.659 Executor task launch worker for task 194 INFO BlockManager: Found block rdd_283_0 locally 20/08/03 12:50:45.659 Executor task launch worker for task 195 INFO BlockManager: Found block rdd_283_1 locally 20/08/03 12:50:45.660 Executor task launch worker for task 194 INFO Executor: Finished task 0.0 in stage 59.0 (TID 194). 2296 bytes result sent to driver 20/08/03 12:50:45.660 Executor task launch worker for task 195 INFO Executor: Finished task 1.0 in stage 59.0 (TID 195). 2296 bytes result sent to driver 20/08/03 12:50:45.660 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 59.0 (TID 196, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.660 Executor task launch worker for task 196 INFO Executor: Running task 2.0 in stage 59.0 (TID 196) 20/08/03 12:50:45.661 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 59.0 (TID 197, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.661 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 59.0 (TID 194) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:45.661 Executor task launch worker for task 197 INFO Executor: Running task 3.0 in stage 59.0 (TID 197) 20/08/03 12:50:45.661 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 59.0 (TID 195) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:45.662 Executor task launch worker for task 196 INFO BlockManager: Found block rdd_283_2 locally 20/08/03 12:50:45.663 Executor task launch worker for task 196 INFO Executor: Finished task 2.0 in stage 59.0 (TID 196). 2296 bytes result sent to driver 20/08/03 12:50:45.664 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 59.0 (TID 196) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:45.665 Executor task launch worker for task 197 INFO BlockManager: Found block rdd_283_3 locally 20/08/03 12:50:45.665 Executor task launch worker for task 197 INFO Executor: Finished task 3.0 in stage 59.0 (TID 197). 2296 bytes result sent to driver 20/08/03 12:50:45.666 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 59.0 (TID 197) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:45.666 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 59.0, whose tasks have all completed, from pool 20/08/03 12:50:45.666 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 59 (treeAggregate at RDDLossFunction.scala:61) finished in 0.014 s 20/08/03 12:50:45.666 dag-scheduler-event-loop INFO DAGScheduler: Job 59 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:45.666 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 59: Stage finished 20/08/03 12:50:45.667 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 59 finished: treeAggregate at RDDLossFunction.scala:61, took 0.015836 s 20/08/03 12:50:45.667 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(116) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:45.668 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_116_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:45.669 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_118 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:45.671 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_118_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:45.671 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_118_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:45.672 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 118 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:45.680 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:45.681 dag-scheduler-event-loop INFO DAGScheduler: Got job 60 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:45.681 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 60 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:45.681 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:45.681 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:45.681 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 60 (MapPartitionsRDD[292] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:45.683 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_119 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:45.684 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_119_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:45.684 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_119_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.685 dag-scheduler-event-loop INFO SparkContext: Created broadcast 119 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:45.685 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 60 (MapPartitionsRDD[292] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:45.685 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 60.0 with 4 tasks 20/08/03 12:50:45.686 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 60.0 (TID 198, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.687 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 60.0 (TID 199, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.687 Executor task launch worker for task 198 INFO Executor: Running task 0.0 in stage 60.0 (TID 198) 20/08/03 12:50:45.687 Executor task launch worker for task 199 INFO Executor: Running task 1.0 in stage 60.0 (TID 199) 20/08/03 12:50:45.689 Executor task launch worker for task 199 INFO BlockManager: Found block rdd_283_1 locally 20/08/03 12:50:45.690 Executor task launch worker for task 199 INFO Executor: Finished task 1.0 in stage 60.0 (TID 199). 2296 bytes result sent to driver 20/08/03 12:50:45.690 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 60.0 (TID 200, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.690 Executor task launch worker for task 200 INFO Executor: Running task 2.0 in stage 60.0 (TID 200) 20/08/03 12:50:45.690 Executor task launch worker for task 198 INFO BlockManager: Found block rdd_283_0 locally 20/08/03 12:50:45.690 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 60.0 (TID 199) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:45.691 Executor task launch worker for task 198 INFO Executor: Finished task 0.0 in stage 60.0 (TID 198). 2296 bytes result sent to driver 20/08/03 12:50:45.692 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 60.0 (TID 201, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:45.692 Executor task launch worker for task 200 INFO BlockManager: Found block rdd_283_2 locally 20/08/03 12:50:45.692 Executor task launch worker for task 201 INFO Executor: Running task 3.0 in stage 60.0 (TID 201) 20/08/03 12:50:45.692 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 60.0 (TID 198) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:45.692 Executor task launch worker for task 200 INFO Executor: Finished task 2.0 in stage 60.0 (TID 200). 2296 bytes result sent to driver 20/08/03 12:50:45.693 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 60.0 (TID 200) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:45.695 Executor task launch worker for task 201 INFO BlockManager: Found block rdd_283_3 locally 20/08/03 12:50:45.696 Executor task launch worker for task 201 INFO Executor: Finished task 3.0 in stage 60.0 (TID 201). 2296 bytes result sent to driver 20/08/03 12:50:45.696 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 60.0 (TID 201) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:45.697 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 60.0, whose tasks have all completed, from pool 20/08/03 12:50:45.697 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 60 (treeAggregate at RDDLossFunction.scala:61) finished in 0.015 s 20/08/03 12:50:45.697 dag-scheduler-event-loop INFO DAGScheduler: Job 60 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:45.697 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 60: Stage finished 20/08/03 12:50:45.697 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 60 finished: treeAggregate at RDDLossFunction.scala:61, took 0.016662 s 20/08/03 12:50:45.698 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(118) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:45.698 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO StrongWolfeLineSearch: Line search t: 0.5508985649472953 fval: 0.7342215028617264 rhs: 0.7358720183663033 cdd: 0.024057870964535294 20/08/03 12:50:45.699 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 0.5509 20/08/03 12:50:45.699 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_118_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:45.699 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.734222 (rel: 0.00225) 0.789814 20/08/03 12:50:45.699 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Converged because max iterations reached 20/08/03 12:50:45.699 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(113) (from destroy at LogisticRegression.scala:804) 20/08/03 12:50:45.700 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MapPartitionsRDD: Removing RDD 283 from persistence list 20/08/03 12:50:45.700 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_113_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 85.0 B, free: 2.1 GiB) 20/08/03 12:50:45.701 block-manager-slave-async-thread-pool-99 INFO BlockManager: Removing RDD 283 20/08/03 12:50:45.767 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_70_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 6.4 KiB, free: 2.1 GiB) 20/08/03 12:50:45.768 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_95_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.769 block-manager-slave-async-thread-pool-10 INFO BlockManager: Removing RDD 112 20/08/03 12:50:45.771 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_82_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.7 KiB, free: 2.1 GiB) 20/08/03 12:50:45.772 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_90_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.7 KiB, free: 2.1 GiB) 20/08/03 12:50:45.772 block-manager-slave-async-thread-pool-31 INFO BlockManager: Removing RDD 214 20/08/03 12:50:45.773 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_103_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.775 block-manager-slave-async-thread-pool-43 INFO BlockManager: Removing RDD 193 20/08/03 12:50:45.776 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_112_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.8 KiB, free: 2.1 GiB) 20/08/03 12:50:45.777 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_64_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 6.4 KiB, free: 2.1 GiB) 20/08/03 12:50:45.777 block-manager-slave-async-thread-pool-54 INFO BlockManager: Removing RDD 256 20/08/03 12:50:45.778 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_109_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:45.779 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_67_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:45.779 block-manager-slave-async-thread-pool-59 INFO BlockManager: Removing RDD 139 20/08/03 12:50:45.780 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_98_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.7 KiB, free: 2.1 GiB) 20/08/03 12:50:45.781 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_97_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.782 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [a96f2526] training finished 20/08/03 12:50:45.783 block-manager-slave-async-thread-pool-81 INFO BlockManager: Removing RDD 166 20/08/03 12:50:45.784 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_76_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 6.4 KiB, free: 2.1 GiB) 20/08/03 12:50:45.785 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_79_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:45.785 block-manager-slave-async-thread-pool-92 INFO BlockManager: Removing RDD 235 20/08/03 12:50:45.786 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_69_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:45.787 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_105_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.789 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_117_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.790 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_111_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:45.791 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_93_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.792 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_81_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:45.793 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_101_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.795 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_115_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.796 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_119_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.796 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_75_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:45.797 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_89_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.798 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_73_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:45.799 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_85_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.800 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_87_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:45.800 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_106_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 6.4 KiB, free: 2.1 GiB) 20/08/03 12:50:45.817 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [259e9803] Stage class: LogisticRegression 20/08/03 12:50:45.817 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [259e9803] Stage uid: logreg_2eb1c67f8c01 20/08/03 12:50:45.838 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [259e9803] training: numPartitions=2 storageLevel=StorageLevel(1 replicas) 20/08/03 12:50:45.838 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [259e9803] {"maxIter":1} 20/08/03 12:50:45.846 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at LogisticRegression.scala:499 20/08/03 12:50:45.846 dag-scheduler-event-loop INFO DAGScheduler: Got job 61 (treeAggregate at LogisticRegression.scala:499) with 2 output partitions 20/08/03 12:50:45.846 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 61 (treeAggregate at LogisticRegression.scala:499) 20/08/03 12:50:45.846 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:45.847 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:45.847 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 61 (MapPartitionsRDD[310] at treeAggregate at LogisticRegression.scala:499), which has no missing parents 20/08/03 12:50:45.850 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_120 stored as values in memory (estimated size 12.6 KiB, free 2.1 GiB) 20/08/03 12:50:45.851 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_120_piece0 stored as bytes in memory (estimated size 6.4 KiB, free 2.1 GiB) 20/08/03 12:50:45.852 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_120_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 6.4 KiB, free: 2.1 GiB) 20/08/03 12:50:45.852 dag-scheduler-event-loop INFO SparkContext: Created broadcast 120 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:45.852 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 61 (MapPartitionsRDD[310] at treeAggregate at LogisticRegression.scala:499) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:45.852 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 61.0 with 2 tasks 20/08/03 12:50:45.853 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 61.0 (TID 202, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:45.853 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 61.0 (TID 203, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:45.854 Executor task launch worker for task 203 INFO Executor: Running task 1.0 in stage 61.0 (TID 203) 20/08/03 12:50:45.854 Executor task launch worker for task 202 INFO Executor: Running task 0.0 in stage 61.0 (TID 202) 20/08/03 12:50:45.858 Executor task launch worker for task 203 INFO MemoryStore: Block rdd_304_1 stored as values in memory (estimated size 3.7 KiB, free 2.1 GiB) 20/08/03 12:50:45.858 Executor task launch worker for task 202 INFO MemoryStore: Block rdd_304_0 stored as values in memory (estimated size 3.7 KiB, free 2.1 GiB) 20/08/03 12:50:45.858 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_304_1 in memory on amp-jenkins-worker-06.amp:34426 (size: 3.7 KiB, free: 2.1 GiB) 20/08/03 12:50:45.858 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_304_0 in memory on amp-jenkins-worker-06.amp:34426 (size: 3.7 KiB, free: 2.1 GiB) 20/08/03 12:50:45.859 Executor task launch worker for task 203 INFO Executor: Finished task 1.0 in stage 61.0 (TID 203). 2543 bytes result sent to driver 20/08/03 12:50:45.859 Executor task launch worker for task 202 INFO Executor: Finished task 0.0 in stage 61.0 (TID 202). 2543 bytes result sent to driver 20/08/03 12:50:45.859 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 61.0 (TID 203) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:45.859 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 61.0 (TID 202) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:45.859 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 61.0, whose tasks have all completed, from pool 20/08/03 12:50:45.860 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 61 (treeAggregate at LogisticRegression.scala:499) finished in 0.013 s 20/08/03 12:50:45.860 dag-scheduler-event-loop INFO DAGScheduler: Job 61 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:45.860 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 61: Stage finished 20/08/03 12:50:45.860 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 61 finished: treeAggregate at LogisticRegression.scala:499, took 0.014433 s 20/08/03 12:50:45.860 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [259e9803] {"numExamples":100} 20/08/03 12:50:45.861 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [259e9803] {"lowestLabelWeight":"34.0"} 20/08/03 12:50:45.861 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [259e9803] {"highestLabelWeight":"66.0"} 20/08/03 12:50:45.861 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [259e9803] {"sumOfWeights":100.0} 20/08/03 12:50:45.861 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [259e9803] {"numClasses":2} 20/08/03 12:50:45.861 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [259e9803] {"numFeatures":1} 20/08/03 12:50:45.861 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_121 stored as values in memory (estimated size 48.0 B, free 2.1 GiB) 20/08/03 12:50:45.863 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_121_piece0 stored as bytes in memory (estimated size 77.0 B, free 2.1 GiB) 20/08/03 12:50:45.863 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_121_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 77.0 B, free: 2.1 GiB) 20/08/03 12:50:45.863 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 121 from broadcast at LogisticRegression.scala:593 20/08/03 12:50:45.864 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_122 stored as values in memory (estimated size 88.0 B, free 2.1 GiB) 20/08/03 12:50:45.865 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_122_piece0 stored as bytes in memory (estimated size 162.0 B, free 2.1 GiB) 20/08/03 12:50:45.865 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_122_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 162.0 B, free: 2.1 GiB) 20/08/03 12:50:45.865 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 122 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:45.866 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticAggregator: Multinomial logistic regression for binary classification yields separate coefficients for positive and negative classes. When no regularization is applied, theresult will be effectively the same as binary logistic regression. When regularizationis applied, multinomial loss will produce a result different from binary loss. 20/08/03 12:50:45.872 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:45.873 dag-scheduler-event-loop INFO DAGScheduler: Got job 62 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:45.873 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 62 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:45.873 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:45.873 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:45.873 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 62 (MapPartitionsRDD[311] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:45.874 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_123 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:45.875 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_123_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:45.875 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_123_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:45.876 dag-scheduler-event-loop INFO SparkContext: Created broadcast 123 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:45.876 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 62 (MapPartitionsRDD[311] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:45.876 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 62.0 with 2 tasks 20/08/03 12:50:45.877 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 62.0 (TID 204, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:45.877 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 62.0 (TID 205, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:45.877 Executor task launch worker for task 205 INFO Executor: Running task 1.0 in stage 62.0 (TID 205) 20/08/03 12:50:45.877 Executor task launch worker for task 204 INFO Executor: Running task 0.0 in stage 62.0 (TID 204) 20/08/03 12:50:45.879 Executor task launch worker for task 205 INFO BlockManager: Found block rdd_304_1 locally 20/08/03 12:50:45.879 Executor task launch worker for task 204 INFO BlockManager: Found block rdd_304_0 locally 20/08/03 12:50:45.879 Executor task launch worker for task 205 INFO Executor: Finished task 1.0 in stage 62.0 (TID 205). 2184 bytes result sent to driver 20/08/03 12:50:45.880 Executor task launch worker for task 204 INFO Executor: Finished task 0.0 in stage 62.0 (TID 204). 2184 bytes result sent to driver 20/08/03 12:50:45.880 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 62.0 (TID 205) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:45.880 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 62.0 (TID 204) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:45.880 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 62.0, whose tasks have all completed, from pool 20/08/03 12:50:45.880 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 62 (treeAggregate at RDDLossFunction.scala:61) finished in 0.007 s 20/08/03 12:50:45.880 dag-scheduler-event-loop INFO DAGScheduler: Job 62 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:45.880 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 62: Stage finished 20/08/03 12:50:45.881 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 62 finished: treeAggregate at RDDLossFunction.scala:61, took 0.008393 s 20/08/03 12:50:45.881 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(122) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:45.882 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_122_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 162.0 B, free: 2.1 GiB) 20/08/03 12:50:45.882 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_124 stored as values in memory (estimated size 88.0 B, free 2.1 GiB) 20/08/03 12:50:45.883 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_124_piece0 stored as bytes in memory (estimated size 169.0 B, free 2.1 GiB) 20/08/03 12:50:45.884 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_124_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 169.0 B, free: 2.1 GiB) 20/08/03 12:50:45.884 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 124 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:45.884 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticAggregator: Multinomial logistic regression for binary classification yields separate coefficients for positive and negative classes. When no regularization is applied, theresult will be effectively the same as binary logistic regression. When regularizationis applied, multinomial loss will produce a result different from binary loss. 20/08/03 12:50:45.890 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:45.891 dag-scheduler-event-loop INFO DAGScheduler: Got job 63 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:45.891 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 63 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:45.891 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:45.891 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:45.891 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 63 (MapPartitionsRDD[312] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:45.892 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_125 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:45.894 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_125_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:45.894 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_125_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:45.894 dag-scheduler-event-loop INFO SparkContext: Created broadcast 125 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:45.895 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 63 (MapPartitionsRDD[312] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:45.895 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 63.0 with 2 tasks 20/08/03 12:50:45.895 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 63.0 (TID 206, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:45.895 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 63.0 (TID 207, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:45.896 Executor task launch worker for task 207 INFO Executor: Running task 1.0 in stage 63.0 (TID 207) 20/08/03 12:50:45.896 Executor task launch worker for task 206 INFO Executor: Running task 0.0 in stage 63.0 (TID 206) 20/08/03 12:50:45.897 Executor task launch worker for task 207 INFO BlockManager: Found block rdd_304_1 locally 20/08/03 12:50:45.897 Executor task launch worker for task 206 INFO BlockManager: Found block rdd_304_0 locally 20/08/03 12:50:45.898 Executor task launch worker for task 207 INFO Executor: Finished task 1.0 in stage 63.0 (TID 207). 2184 bytes result sent to driver 20/08/03 12:50:45.898 Executor task launch worker for task 206 INFO Executor: Finished task 0.0 in stage 63.0 (TID 206). 2184 bytes result sent to driver 20/08/03 12:50:45.898 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 63.0 (TID 207) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:45.898 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 63.0 (TID 206) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:45.899 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 63.0, whose tasks have all completed, from pool 20/08/03 12:50:45.899 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 63 (treeAggregate at RDDLossFunction.scala:61) finished in 0.007 s 20/08/03 12:50:45.899 dag-scheduler-event-loop INFO DAGScheduler: Job 63 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:45.899 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 63: Stage finished 20/08/03 12:50:45.900 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 63 finished: treeAggregate at RDDLossFunction.scala:61, took 0.008997 s 20/08/03 12:50:45.900 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(124) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:45.900 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 2.787 20/08/03 12:50:45.901 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.483031 (rel: 0.247) 0.0670723 20/08/03 12:50:45.901 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_124_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 169.0 B, free: 2.1 GiB) 20/08/03 12:50:45.901 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Converged because max iterations reached 20/08/03 12:50:45.901 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(121) (from destroy at LogisticRegression.scala:804) 20/08/03 12:50:45.901 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MapPartitionsRDD: Removing RDD 304 from persistence list 20/08/03 12:50:45.901 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_121_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 77.0 B, free: 2.1 GiB) 20/08/03 12:50:45.902 block-manager-slave-async-thread-pool-4 INFO BlockManager: Removing RDD 304 20/08/03 12:50:45.978 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [259e9803] training finished 20/08/03 12:50:46.098 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: ===== FINISHED o.a.s.ml.classification.LogisticRegressionSuite: 'check summary types for binary and multiclass' ===== 20/08/03 12:50:46.099 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: ===== TEST OUTPUT FOR o.a.s.ml.classification.LogisticRegressionSuite: 'setThreshold, getThreshold' ===== 20/08/03 12:50:46.107 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: ===== FINISHED o.a.s.ml.classification.LogisticRegressionSuite: 'setThreshold, getThreshold' ===== 20/08/03 12:50:46.108 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: ===== TEST OUTPUT FOR o.a.s.ml.classification.LogisticRegressionSuite: 'thresholds prediction' ===== 20/08/03 12:50:46.137 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [3f39cd31] Stage class: LogisticRegression 20/08/03 12:50:46.137 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [3f39cd31] Stage uid: logreg_8ea4fe2f34f3 20/08/03 12:50:46.157 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [3f39cd31] training: numPartitions=2 storageLevel=StorageLevel(1 replicas) 20/08/03 12:50:46.157 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [3f39cd31] {} 20/08/03 12:50:46.165 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at LogisticRegression.scala:499 20/08/03 12:50:46.166 dag-scheduler-event-loop INFO DAGScheduler: Got job 64 (treeAggregate at LogisticRegression.scala:499) with 2 output partitions 20/08/03 12:50:46.166 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 64 (treeAggregate at LogisticRegression.scala:499) 20/08/03 12:50:46.166 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:46.166 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:46.167 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 64 (MapPartitionsRDD[356] at treeAggregate at LogisticRegression.scala:499), which has no missing parents 20/08/03 12:50:46.171 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_126 stored as values in memory (estimated size 12.6 KiB, free 2.1 GiB) 20/08/03 12:50:46.173 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_126_piece0 stored as bytes in memory (estimated size 6.4 KiB, free 2.1 GiB) 20/08/03 12:50:46.173 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_126_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 6.4 KiB, free: 2.1 GiB) 20/08/03 12:50:46.173 dag-scheduler-event-loop INFO SparkContext: Created broadcast 126 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:46.174 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 64 (MapPartitionsRDD[356] at treeAggregate at LogisticRegression.scala:499) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:46.174 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 64.0 with 2 tasks 20/08/03 12:50:46.174 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 64.0 (TID 208, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.175 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 64.0 (TID 209, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.175 Executor task launch worker for task 208 INFO Executor: Running task 0.0 in stage 64.0 (TID 208) 20/08/03 12:50:46.175 Executor task launch worker for task 209 INFO Executor: Running task 1.0 in stage 64.0 (TID 209) 20/08/03 12:50:46.179 Executor task launch worker for task 209 INFO MemoryStore: Block rdd_350_1 stored as values in memory (estimated size 3.7 KiB, free 2.1 GiB) 20/08/03 12:50:46.180 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_350_1 in memory on amp-jenkins-worker-06.amp:34426 (size: 3.7 KiB, free: 2.1 GiB) 20/08/03 12:50:46.180 Executor task launch worker for task 209 INFO Executor: Finished task 1.0 in stage 64.0 (TID 209). 2543 bytes result sent to driver 20/08/03 12:50:46.181 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 64.0 (TID 209) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:46.182 Executor task launch worker for task 208 INFO MemoryStore: Block rdd_350_0 stored as values in memory (estimated size 3.7 KiB, free 2.1 GiB) 20/08/03 12:50:46.182 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_350_0 in memory on amp-jenkins-worker-06.amp:34426 (size: 3.7 KiB, free: 2.1 GiB) 20/08/03 12:50:46.184 Executor task launch worker for task 208 INFO Executor: Finished task 0.0 in stage 64.0 (TID 208). 2543 bytes result sent to driver 20/08/03 12:50:46.185 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 64.0 (TID 208) in 11 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:46.185 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 64.0, whose tasks have all completed, from pool 20/08/03 12:50:46.185 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 64 (treeAggregate at LogisticRegression.scala:499) finished in 0.018 s 20/08/03 12:50:46.185 dag-scheduler-event-loop INFO DAGScheduler: Job 64 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:46.185 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 64: Stage finished 20/08/03 12:50:46.186 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 64 finished: treeAggregate at LogisticRegression.scala:499, took 0.020207 s 20/08/03 12:50:46.186 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [3f39cd31] {"numExamples":100} 20/08/03 12:50:46.187 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [3f39cd31] {"lowestLabelWeight":"34.0"} 20/08/03 12:50:46.187 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [3f39cd31] {"highestLabelWeight":"66.0"} 20/08/03 12:50:46.187 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [3f39cd31] {"sumOfWeights":100.0} 20/08/03 12:50:46.187 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [3f39cd31] {"numClasses":2} 20/08/03 12:50:46.187 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [3f39cd31] {"numFeatures":1} 20/08/03 12:50:46.187 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_127 stored as values in memory (estimated size 48.0 B, free 2.1 GiB) 20/08/03 12:50:46.189 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_127_piece0 stored as bytes in memory (estimated size 77.0 B, free 2.1 GiB) 20/08/03 12:50:46.189 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_127_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 77.0 B, free: 2.1 GiB) 20/08/03 12:50:46.190 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 127 from broadcast at LogisticRegression.scala:593 20/08/03 12:50:46.190 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_128 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:46.192 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_128_piece0 stored as bytes in memory (estimated size 153.0 B, free 2.1 GiB) 20/08/03 12:50:46.192 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_128_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 153.0 B, free: 2.1 GiB) 20/08/03 12:50:46.192 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 128 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:46.199 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:46.200 dag-scheduler-event-loop INFO DAGScheduler: Got job 65 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:46.200 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 65 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:46.200 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:46.200 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:46.200 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 65 (MapPartitionsRDD[357] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:46.202 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_129 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:46.203 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_129_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:46.203 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_129_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:46.203 dag-scheduler-event-loop INFO SparkContext: Created broadcast 129 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:46.204 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 65 (MapPartitionsRDD[357] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:46.204 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 65.0 with 2 tasks 20/08/03 12:50:46.205 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 65.0 (TID 210, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.205 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 65.0 (TID 211, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.205 Executor task launch worker for task 210 INFO Executor: Running task 0.0 in stage 65.0 (TID 210) 20/08/03 12:50:46.205 Executor task launch worker for task 211 INFO Executor: Running task 1.0 in stage 65.0 (TID 211) 20/08/03 12:50:46.207 Executor task launch worker for task 210 INFO BlockManager: Found block rdd_350_0 locally 20/08/03 12:50:46.207 Executor task launch worker for task 211 INFO BlockManager: Found block rdd_350_1 locally 20/08/03 12:50:46.208 Executor task launch worker for task 210 INFO Executor: Finished task 0.0 in stage 65.0 (TID 210). 2168 bytes result sent to driver 20/08/03 12:50:46.208 Executor task launch worker for task 211 INFO Executor: Finished task 1.0 in stage 65.0 (TID 211). 2168 bytes result sent to driver 20/08/03 12:50:46.209 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 65.0 (TID 210) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:46.209 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 65.0 (TID 211) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:46.209 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 65.0, whose tasks have all completed, from pool 20/08/03 12:50:46.209 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 65 (treeAggregate at RDDLossFunction.scala:61) finished in 0.008 s 20/08/03 12:50:46.209 dag-scheduler-event-loop INFO DAGScheduler: Job 65 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:46.209 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 65: Stage finished 20/08/03 12:50:46.210 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 65 finished: treeAggregate at RDDLossFunction.scala:61, took 0.010428 s 20/08/03 12:50:46.210 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(128) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:46.211 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_128_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 153.0 B, free: 2.1 GiB) 20/08/03 12:50:46.211 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_130 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:46.212 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_130_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:46.212 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_130_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:46.213 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 130 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:46.219 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:46.220 dag-scheduler-event-loop INFO DAGScheduler: Got job 66 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:46.220 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 66 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:46.220 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:46.220 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:46.220 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 66 (MapPartitionsRDD[358] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:46.222 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_131 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:46.223 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_131_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:46.223 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_131_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:46.223 dag-scheduler-event-loop INFO SparkContext: Created broadcast 131 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:46.224 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 66 (MapPartitionsRDD[358] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:46.224 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 66.0 with 2 tasks 20/08/03 12:50:46.224 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 66.0 (TID 212, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.225 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 66.0 (TID 213, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.225 Executor task launch worker for task 212 INFO Executor: Running task 0.0 in stage 66.0 (TID 212) 20/08/03 12:50:46.225 Executor task launch worker for task 213 INFO Executor: Running task 1.0 in stage 66.0 (TID 213) 20/08/03 12:50:46.227 Executor task launch worker for task 213 INFO BlockManager: Found block rdd_350_1 locally 20/08/03 12:50:46.227 Executor task launch worker for task 212 INFO BlockManager: Found block rdd_350_0 locally 20/08/03 12:50:46.227 Executor task launch worker for task 213 INFO Executor: Finished task 1.0 in stage 66.0 (TID 213). 2168 bytes result sent to driver 20/08/03 12:50:46.227 Executor task launch worker for task 212 INFO Executor: Finished task 0.0 in stage 66.0 (TID 212). 2168 bytes result sent to driver 20/08/03 12:50:46.228 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 66.0 (TID 213) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:46.228 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 66.0 (TID 212) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:46.228 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 66.0, whose tasks have all completed, from pool 20/08/03 12:50:46.228 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 66 (treeAggregate at RDDLossFunction.scala:61) finished in 0.007 s 20/08/03 12:50:46.229 dag-scheduler-event-loop INFO DAGScheduler: Job 66 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:46.229 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 66: Stage finished 20/08/03 12:50:46.229 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 66 finished: treeAggregate at RDDLossFunction.scala:61, took 0.009551 s 20/08/03 12:50:46.229 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(130) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:46.230 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 3.941 20/08/03 12:50:46.230 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.493391 (rel: 0.230) 0.0621158 20/08/03 12:50:46.230 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_130_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:46.230 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_132 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:46.232 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_132_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:46.232 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_132_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:46.232 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 132 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:46.241 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:46.241 dag-scheduler-event-loop INFO DAGScheduler: Got job 67 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:46.241 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 67 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:46.241 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:46.242 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:46.242 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 67 (MapPartitionsRDD[359] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:46.243 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_133 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:46.245 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_133_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:46.245 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_133_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:46.245 dag-scheduler-event-loop INFO SparkContext: Created broadcast 133 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:46.245 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 67 (MapPartitionsRDD[359] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:46.246 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 67.0 with 2 tasks 20/08/03 12:50:46.246 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 67.0 (TID 214, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.247 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 67.0 (TID 215, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.247 Executor task launch worker for task 214 INFO Executor: Running task 0.0 in stage 67.0 (TID 214) 20/08/03 12:50:46.247 Executor task launch worker for task 215 INFO Executor: Running task 1.0 in stage 67.0 (TID 215) 20/08/03 12:50:46.249 Executor task launch worker for task 215 INFO BlockManager: Found block rdd_350_1 locally 20/08/03 12:50:46.249 Executor task launch worker for task 214 INFO BlockManager: Found block rdd_350_0 locally 20/08/03 12:50:46.249 Executor task launch worker for task 215 INFO Executor: Finished task 1.0 in stage 67.0 (TID 215). 2168 bytes result sent to driver 20/08/03 12:50:46.249 Executor task launch worker for task 214 INFO Executor: Finished task 0.0 in stage 67.0 (TID 214). 2168 bytes result sent to driver 20/08/03 12:50:46.250 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 67.0 (TID 215) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:46.250 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 67.0 (TID 214) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:46.250 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 67.0, whose tasks have all completed, from pool 20/08/03 12:50:46.250 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 67 (treeAggregate at RDDLossFunction.scala:61) finished in 0.008 s 20/08/03 12:50:46.250 dag-scheduler-event-loop INFO DAGScheduler: Job 67 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:46.250 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 67: Stage finished 20/08/03 12:50:46.251 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 67 finished: treeAggregate at RDDLossFunction.scala:61, took 0.009957 s 20/08/03 12:50:46.251 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(132) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:46.252 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:46.252 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.478652 (rel: 0.0299) 0.0230034 20/08/03 12:50:46.252 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_132_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:46.253 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_134 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:46.254 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_134_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:46.254 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_134_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:46.255 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 134 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:46.261 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:46.261 dag-scheduler-event-loop INFO DAGScheduler: Got job 68 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:46.262 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 68 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:46.262 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:46.262 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:46.262 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 68 (MapPartitionsRDD[360] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:46.264 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_135 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:46.265 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_135_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:46.265 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_135_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:46.265 dag-scheduler-event-loop INFO SparkContext: Created broadcast 135 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:46.266 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 68 (MapPartitionsRDD[360] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:46.266 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 68.0 with 2 tasks 20/08/03 12:50:46.266 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 68.0 (TID 216, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.267 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 68.0 (TID 217, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.267 Executor task launch worker for task 216 INFO Executor: Running task 0.0 in stage 68.0 (TID 216) 20/08/03 12:50:46.267 Executor task launch worker for task 217 INFO Executor: Running task 1.0 in stage 68.0 (TID 217) 20/08/03 12:50:46.269 Executor task launch worker for task 217 INFO BlockManager: Found block rdd_350_1 locally 20/08/03 12:50:46.269 Executor task launch worker for task 216 INFO BlockManager: Found block rdd_350_0 locally 20/08/03 12:50:46.269 Executor task launch worker for task 217 INFO Executor: Finished task 1.0 in stage 68.0 (TID 217). 2168 bytes result sent to driver 20/08/03 12:50:46.270 Executor task launch worker for task 216 INFO Executor: Finished task 0.0 in stage 68.0 (TID 216). 2168 bytes result sent to driver 20/08/03 12:50:46.270 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 68.0 (TID 217) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:46.270 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 68.0 (TID 216) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:46.270 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 68.0, whose tasks have all completed, from pool 20/08/03 12:50:46.271 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 68 (treeAggregate at RDDLossFunction.scala:61) finished in 0.007 s 20/08/03 12:50:46.271 dag-scheduler-event-loop INFO DAGScheduler: Job 68 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:46.271 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 68: Stage finished 20/08/03 12:50:46.271 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 68 finished: treeAggregate at RDDLossFunction.scala:61, took 0.010015 s 20/08/03 12:50:46.271 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(134) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:46.272 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:46.272 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_134_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:46.272 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.475811 (rel: 0.00594) 0.00507866 20/08/03 12:50:46.273 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_136 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:46.274 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_136_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:46.274 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_136_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:46.274 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 136 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:46.280 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:46.281 dag-scheduler-event-loop INFO DAGScheduler: Got job 69 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:46.281 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 69 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:46.281 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:46.281 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:46.282 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 69 (MapPartitionsRDD[361] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:46.283 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_137 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:46.284 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_137_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:46.284 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_137_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:46.284 dag-scheduler-event-loop INFO SparkContext: Created broadcast 137 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:46.284 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 69 (MapPartitionsRDD[361] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:46.285 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 69.0 with 2 tasks 20/08/03 12:50:46.285 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 69.0 (TID 218, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.285 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 69.0 (TID 219, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.286 Executor task launch worker for task 218 INFO Executor: Running task 0.0 in stage 69.0 (TID 218) 20/08/03 12:50:46.286 Executor task launch worker for task 219 INFO Executor: Running task 1.0 in stage 69.0 (TID 219) 20/08/03 12:50:46.287 Executor task launch worker for task 219 INFO BlockManager: Found block rdd_350_1 locally 20/08/03 12:50:46.287 Executor task launch worker for task 218 INFO BlockManager: Found block rdd_350_0 locally 20/08/03 12:50:46.288 Executor task launch worker for task 219 INFO Executor: Finished task 1.0 in stage 69.0 (TID 219). 2168 bytes result sent to driver 20/08/03 12:50:46.288 Executor task launch worker for task 218 INFO Executor: Finished task 0.0 in stage 69.0 (TID 218). 2168 bytes result sent to driver 20/08/03 12:50:46.288 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 69.0 (TID 219) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:46.288 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 69.0 (TID 218) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:46.288 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 69.0, whose tasks have all completed, from pool 20/08/03 12:50:46.289 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 69 (treeAggregate at RDDLossFunction.scala:61) finished in 0.007 s 20/08/03 12:50:46.289 dag-scheduler-event-loop INFO DAGScheduler: Job 69 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:46.289 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 69: Stage finished 20/08/03 12:50:46.289 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 69 finished: treeAggregate at RDDLossFunction.scala:61, took 0.008401 s 20/08/03 12:50:46.289 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(136) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:46.290 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:46.290 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_136_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:46.290 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.475650 (rel: 0.000337) 0.00106784 20/08/03 12:50:46.291 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_138 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:46.292 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_138_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:46.292 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_138_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:46.292 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 138 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:46.299 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:46.299 dag-scheduler-event-loop INFO DAGScheduler: Got job 70 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:46.299 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 70 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:46.299 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:46.299 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:46.300 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 70 (MapPartitionsRDD[362] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:46.301 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_139 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:46.302 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_139_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:46.302 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_139_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:46.302 dag-scheduler-event-loop INFO SparkContext: Created broadcast 139 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:46.302 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 70 (MapPartitionsRDD[362] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:46.302 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 70.0 with 2 tasks 20/08/03 12:50:46.303 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 70.0 (TID 220, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.303 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 70.0 (TID 221, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.303 Executor task launch worker for task 220 INFO Executor: Running task 0.0 in stage 70.0 (TID 220) 20/08/03 12:50:46.303 Executor task launch worker for task 221 INFO Executor: Running task 1.0 in stage 70.0 (TID 221) 20/08/03 12:50:46.305 Executor task launch worker for task 220 INFO BlockManager: Found block rdd_350_0 locally 20/08/03 12:50:46.306 Executor task launch worker for task 220 INFO Executor: Finished task 0.0 in stage 70.0 (TID 220). 2168 bytes result sent to driver 20/08/03 12:50:46.306 Executor task launch worker for task 221 INFO BlockManager: Found block rdd_350_1 locally 20/08/03 12:50:46.306 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 70.0 (TID 220) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:46.306 Executor task launch worker for task 221 INFO Executor: Finished task 1.0 in stage 70.0 (TID 221). 2168 bytes result sent to driver 20/08/03 12:50:46.307 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 70.0 (TID 221) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:46.307 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 70.0, whose tasks have all completed, from pool 20/08/03 12:50:46.307 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 70 (treeAggregate at RDDLossFunction.scala:61) finished in 0.007 s 20/08/03 12:50:46.307 dag-scheduler-event-loop INFO DAGScheduler: Job 70 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:46.307 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 70: Stage finished 20/08/03 12:50:46.308 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 70 finished: treeAggregate at RDDLossFunction.scala:61, took 0.008596 s 20/08/03 12:50:46.308 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(138) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:46.308 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:46.308 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_138_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:46.308 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.475647 (rel: 6.99e-06) 0.000474829 20/08/03 12:50:46.309 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_140 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:46.311 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_140_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:46.311 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_140_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:46.311 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 140 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:46.319 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:46.320 dag-scheduler-event-loop INFO DAGScheduler: Got job 71 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:46.320 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 71 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:46.320 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:46.320 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:46.320 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 71 (MapPartitionsRDD[363] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:46.322 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_141 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:46.323 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_141_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:46.323 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_141_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:46.323 dag-scheduler-event-loop INFO SparkContext: Created broadcast 141 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:46.324 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 71 (MapPartitionsRDD[363] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:46.324 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 71.0 with 2 tasks 20/08/03 12:50:46.324 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 71.0 (TID 222, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.325 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 71.0 (TID 223, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.325 Executor task launch worker for task 223 INFO Executor: Running task 1.0 in stage 71.0 (TID 223) 20/08/03 12:50:46.325 Executor task launch worker for task 222 INFO Executor: Running task 0.0 in stage 71.0 (TID 222) 20/08/03 12:50:46.328 Executor task launch worker for task 223 INFO BlockManager: Found block rdd_350_1 locally 20/08/03 12:50:46.328 Executor task launch worker for task 223 INFO Executor: Finished task 1.0 in stage 71.0 (TID 223). 2168 bytes result sent to driver 20/08/03 12:50:46.328 Executor task launch worker for task 222 INFO BlockManager: Found block rdd_350_0 locally 20/08/03 12:50:46.329 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 71.0 (TID 223) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:46.330 Executor task launch worker for task 222 INFO Executor: Finished task 0.0 in stage 71.0 (TID 222). 2211 bytes result sent to driver 20/08/03 12:50:46.330 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 71.0 (TID 222) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:46.330 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 71.0, whose tasks have all completed, from pool 20/08/03 12:50:46.331 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 71 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:46.331 dag-scheduler-event-loop INFO DAGScheduler: Job 71 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:46.331 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 71: Stage finished 20/08/03 12:50:46.331 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 71 finished: treeAggregate at RDDLossFunction.scala:61, took 0.011806 s 20/08/03 12:50:46.331 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(140) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:46.332 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:46.332 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_140_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:46.332 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.475647 (rel: 1.30e-06) 3.45553e-06 20/08/03 12:50:46.334 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_142 stored as values in memory (estimated size 72.0 B, free 2.1 GiB) 20/08/03 12:50:46.336 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_142_piece0 stored as bytes in memory (estimated size 154.0 B, free 2.1 GiB) 20/08/03 12:50:46.336 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_142_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:46.336 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 142 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:46.342 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:46.343 dag-scheduler-event-loop INFO DAGScheduler: Got job 72 (treeAggregate at RDDLossFunction.scala:61) with 2 output partitions 20/08/03 12:50:46.343 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 72 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:46.343 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:46.343 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:46.343 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 72 (MapPartitionsRDD[364] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:46.344 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_143 stored as values in memory (estimated size 11.7 KiB, free 2.1 GiB) 20/08/03 12:50:46.345 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_143_piece0 stored as bytes in memory (estimated size 5.9 KiB, free 2.1 GiB) 20/08/03 12:50:46.346 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_143_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:46.346 dag-scheduler-event-loop INFO SparkContext: Created broadcast 143 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:46.346 dag-scheduler-event-loop INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 72 (MapPartitionsRDD[364] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1)) 20/08/03 12:50:46.346 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 72.0 with 2 tasks 20/08/03 12:50:46.347 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 72.0 (TID 224, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.347 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 72.0 (TID 225, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 13142 bytes) 20/08/03 12:50:46.347 Executor task launch worker for task 225 INFO Executor: Running task 1.0 in stage 72.0 (TID 225) 20/08/03 12:50:46.347 Executor task launch worker for task 224 INFO Executor: Running task 0.0 in stage 72.0 (TID 224) 20/08/03 12:50:46.349 Executor task launch worker for task 225 INFO BlockManager: Found block rdd_350_1 locally 20/08/03 12:50:46.350 Executor task launch worker for task 225 INFO Executor: Finished task 1.0 in stage 72.0 (TID 225). 2168 bytes result sent to driver 20/08/03 12:50:46.350 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 72.0 (TID 225) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/2) 20/08/03 12:50:46.351 Executor task launch worker for task 224 INFO BlockManager: Found block rdd_350_0 locally 20/08/03 12:50:46.352 Executor task launch worker for task 224 INFO Executor: Finished task 0.0 in stage 72.0 (TID 224). 2211 bytes result sent to driver 20/08/03 12:50:46.353 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 72.0 (TID 224) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (2/2) 20/08/03 12:50:46.353 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 72.0, whose tasks have all completed, from pool 20/08/03 12:50:46.353 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 72 (treeAggregate at RDDLossFunction.scala:61) finished in 0.009 s 20/08/03 12:50:46.353 dag-scheduler-event-loop INFO DAGScheduler: Job 72 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:46.353 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 72: Stage finished 20/08/03 12:50:46.353 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 72 finished: treeAggregate at RDDLossFunction.scala:61, took 0.010980 s 20/08/03 12:50:46.354 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(142) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:46.354 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:46.355 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_142_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 154.0 B, free: 2.1 GiB) 20/08/03 12:50:46.355 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.475647 (rel: 9.23e-11) 6.96515e-08 20/08/03 12:50:46.355 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Converged because gradient converged 20/08/03 12:50:46.355 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(127) (from destroy at LogisticRegression.scala:804) 20/08/03 12:50:46.356 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MapPartitionsRDD: Removing RDD 350 from persistence list 20/08/03 12:50:46.356 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_127_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 77.0 B, free: 2.1 GiB) 20/08/03 12:50:46.356 block-manager-slave-async-thread-pool-37 INFO BlockManager: Removing RDD 350 20/08/03 12:50:46.430 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [3f39cd31] training finished 20/08/03 12:50:46.491 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 9.152483 ms 20/08/03 12:50:46.527 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO StateStoreCoordinatorRef: Registered StateStoreCoordinator endpoint 20/08/03 12:50:46.534 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: StartStream(ProcessingTimeTrigger(0),org.apache.spark.util.SystemClock@6016d2fb,Map(),null) 20/08/03 12:50:46.556 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Checkpoint root /home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-9ccc101c-bae5-4c74-9f05-46166fcec276 resolved to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-9ccc101c-bae5-4c74-9f05-46166fcec276. 20/08/03 12:50:46.603 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-9ccc101c-bae5-4c74-9f05-46166fcec276/metadata using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-9ccc101c-bae5-4c74-9f05-46166fcec276/.metadata.d53258a8-7c08-42af-b2d1-41256149d6da.tmp 20/08/03 12:50:46.690 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-9ccc101c-bae5-4c74-9f05-46166fcec276/.metadata.d53258a8-7c08-42af-b2d1-41256149d6da.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-9ccc101c-bae5-4c74-9f05-46166fcec276/metadata 20/08/03 12:50:46.730 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Starting [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a]. Use file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-9ccc101c-bae5-4c74-9f05-46166fcec276 to store the query checkpoint. 20/08/03 12:50:46.737 stream execution thread for [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] INFO MicroBatchExecution: Reading table [org.apache.spark.sql.execution.streaming.MemoryStreamTable@77a8b912] from DataSourceV2 named 'memory' [org.apache.spark.sql.execution.streaming.MemoryStreamTableProvider$@490dc5fd] 20/08/03 12:50:46.746 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: AddData to MemoryStream[_1#773,_2#774]: (1.0,[1.1419053154730547]),(0.0,[0.9194079489827879]),(0.0,[-0.9498666368908959]),(1.0,[-1.1069902863993377]),(0.0,[0.2809776380727795]),(1.0,[0.6846227956326554]),(1.0,[-0.8172214073987268]),(0.0,[-1.3966434026780434]),(1.0,[-0.19094451307087512]),(1.0,[1.4862133923906502]),(1.0,[0.8023071496873626]),(0.0,[-0.12151292466549345]),(1.0,[1.4105062239438624]),(0.0,[-0.6402327822135738]),(0.0,[-1.2096444592532913]),(1.0,[0.35375769787202876]),(0.0,[-0.4903496491990076]),(0.0,[0.5507215382743629]),(1.0,[-1.2035510019650835]),(1.0,[0.3210160806416416]),(1.0,[1.5511476388671834]),(1.0,[0.43853028624710505]),(1.0,[0.4815980608245389]),(1.0,[1.5196310789680683]),(0.0,[-0.2768317291873249]),(1.0,[-0.08393897849486337]),(1.0,[1.255833005788796]),(0.0,[-0.3252727938665772]),(0.0,[-0.17329033306108363]),(0.0,[-1.8585851445864527]),(1.0,[1.4238069456328435]),(0.0,[-1.363726024075023]),(0.0,[-1.964666098753878]),(1.0,[-0.9185948439341892]),(0.0,[-2.548887393384806]),(0.0,[-1.6309606578419305]),(1.0,[-0.12200477461989162]),(1.0,[1.289159071801577]),(1.0,[-0.2691388556559934]),(1.0,[0.2574914085090889]),(0.0,[-0.3199143760045327]),(0.0,[-1.7684998592513064]),(1.0,[-0.4834503128592458]),(1.0,[-0.5099904653893699]),(1.0,[1.1166733769661994]),(1.0,[-0.04094720151728288]),(0.0,[-1.1076715169200795]),(1.0,[1.8623214176471945]),(1.0,[1.1457411377091524]),(1.0,[-1.0586772048930921]),(1.0,[1.0725991339400673]),(1.0,[-1.9317441520296659]),(1.0,[0.30102521611534994]),(1.0,[0.2475231582804265]),(1.0,[1.406156849249087]),(0.0,[-1.5202207203569256]),(1.0,[0.2709294126920897]),(1.0,[0.561249284813777]),(0.0,[-0.5298295780368607]),(0.0,[0.5390221914988275]),(1.0,[2.2123402141787243]),(1.0,[-0.6329335687728442]),(0.0,[-1.8831759122084633]),(1.0,[0.3865659853763343]),(1.0,[0.32582927090649455]),(0.0,[-0.9013043195000002]),(1.0,[-0.002680308907617573]),(1.0,[-0.4739592549853249]),(1.0,[-0.5479781547659026]),(1.0,[-0.01910014847196348]),(1.0,[1.6468163882596327]),(0.0,[-1.107062592215791]),(1.0,[0.5938103926672539]),(0.0,[-0.15566462108511642]),(0.0,[0.6632872929286855]),(1.0,[1.226793360688623]),(1.0,[0.8839698437730904]),(0.0,[0.22172454670212935]),(1.0,[0.9197020859698617]),(0.0,[-0.7393758185888677]),(1.0,[0.803517749531419]),(1.0,[-0.2539417447630359]),(1.0,[-0.7638388605060555]),(0.0,[-1.8645567427274516]),(0.0,[-1.861306200027518]),(0.0,[-0.576599881116305]),(1.0,[-0.40899380621224757]),(1.0,[0.24846093761654187]),(1.0,[-0.48091295490277447]),(1.0,[0.44621205735391023]),(1.0,[-0.4465888888803913]),(1.0,[0.045638687865053575]),(1.0,[0.7045663273135641]),(1.0,[-0.2718240183671583]),(1.0,[0.08074877915238832]),(1.0,[1.2590964696340183]),(1.0,[0.7635098382407334]),(1.0,[1.7220810801509723]),(1.0,[0.14595005405372477]),(0.0,[-0.9946630124621867]) 20/08/03 12:50:46.749 stream execution thread for [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] INFO MicroBatchExecution: Starting new streaming query. 20/08/03 12:50:46.752 stream execution thread for [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] INFO MicroBatchExecution: Stream started from {} 20/08/03 12:50:46.770 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 17.667804 ms 20/08/03 12:50:46.776 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: CheckAnswerByFunc 20/08/03 12:50:46.782 stream execution thread for [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] INFO MicroBatchExecution: Streaming query made progress: { "id" : "0739a101-ca0f-4949-94da-a742f548541b", "runId" : "da58b791-8868-476d-a542-73c733f1113a", "name" : null, "timestamp" : "2020-08-03T19:50:46.746Z", "batchId" : 0, "numInputRows" : 0, "processedRowsPerSecond" : 0.0, "durationMs" : { "latestOffset" : 0, "triggerExecution" : 11 }, "stateOperators" : [ ], "sources" : [ { "description" : "MemoryStream[_1#773,_2#774]", "startOffset" : null, "endOffset" : null, "numInputRows" : 0, "processedRowsPerSecond" : 0.0 } ], "sink" : { "description" : "MemorySink", "numOutputRows" : -1 } } 20/08/03 12:50:46.805 stream execution thread for [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-9ccc101c-bae5-4c74-9f05-46166fcec276/offsets/0 using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-9ccc101c-bae5-4c74-9f05-46166fcec276/offsets/.0.6b010e14-7963-4997-8cb4-cca361ebe7ea.tmp 20/08/03 12:50:46.839 stream execution thread for [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-9ccc101c-bae5-4c74-9f05-46166fcec276/offsets/.0.6b010e14-7963-4997-8cb4-cca361ebe7ea.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-9ccc101c-bae5-4c74-9f05-46166fcec276/offsets/0 20/08/03 12:50:46.840 stream execution thread for [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] INFO MicroBatchExecution: Committed offsets for batch 0. Metadata OffsetSeqMetadata(0,1596484246792,Map(spark.sql.streaming.stateStore.providerClass -> org.apache.spark.sql.execution.streaming.state.HDFSBackedStateStoreProvider, spark.sql.streaming.join.stateFormatVersion -> 2, spark.sql.streaming.flatMapGroupsWithState.stateFormatVersion -> 2, spark.sql.streaming.multipleWatermarkPolicy -> min, spark.sql.streaming.aggregation.stateFormatVersion -> 2, spark.sql.shuffle.partitions -> 5)) 20/08/03 12:50:46.953 stream execution thread for [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] INFO CodeGenerator: Code generated in 16.858132 ms 20/08/03 12:50:46.968 stream execution thread for [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] INFO WriteToDataSourceV2Exec: Start processing data source write support: org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@7bd444cb. The input RDD has 1 partitions. 20/08/03 12:50:47.025 stream execution thread for [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] INFO SparkContext: Starting job: testStream$ at LogisticRegressionSuite.scala:39 20/08/03 12:50:47.026 dag-scheduler-event-loop INFO DAGScheduler: Got job 73 (testStream$ at LogisticRegressionSuite.scala:39) with 1 output partitions 20/08/03 12:50:47.027 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 73 (testStream$ at LogisticRegressionSuite.scala:39) 20/08/03 12:50:47.027 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.027 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.027 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 73 (MapPartitionsRDD[380] at testStream$ at LogisticRegressionSuite.scala:39), which has no missing parents 20/08/03 12:50:47.030 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_144 stored as values in memory (estimated size 24.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.031 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_144_piece0 stored as bytes in memory (estimated size 10.4 KiB, free 2.1 GiB) 20/08/03 12:50:47.032 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_144_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.4 KiB, free: 2.1 GiB) 20/08/03 12:50:47.032 dag-scheduler-event-loop INFO SparkContext: Created broadcast 144 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.033 dag-scheduler-event-loop INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 73 (MapPartitionsRDD[380] at testStream$ at LogisticRegressionSuite.scala:39) (first 15 tasks are for partitions Vector(0)) 20/08/03 12:50:47.033 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 73.0 with 1 tasks 20/08/03 12:50:47.035 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 73.0 (TID 226, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 18076 bytes) 20/08/03 12:50:47.035 Executor task launch worker for task 226 INFO Executor: Running task 0.0 in stage 73.0 (TID 226) 20/08/03 12:50:47.103 Executor task launch worker for task 226 INFO CodeGenerator: Code generated in 9.826682 ms 20/08/03 12:50:47.113 Executor task launch worker for task 226 INFO DataWritingSparkTask: Commit authorized for partition 0 (task 226, attempt 0, stage 73.0) 20/08/03 12:50:47.114 Executor task launch worker for task 226 INFO DataWritingSparkTask: Committed partition 0 (task 226, attempt 0, stage 73.0) 20/08/03 12:50:47.117 Executor task launch worker for task 226 INFO Executor: Finished task 0.0 in stage 73.0 (TID 226). 6515 bytes result sent to driver 20/08/03 12:50:47.118 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 73.0 (TID 226) in 85 ms on amp-jenkins-worker-06.amp (executor driver) (1/1) 20/08/03 12:50:47.118 task-result-getter-2 INFO TaskSchedulerImpl: Removed TaskSet 73.0, whose tasks have all completed, from pool 20/08/03 12:50:47.118 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 73 (testStream$ at LogisticRegressionSuite.scala:39) finished in 0.090 s 20/08/03 12:50:47.118 dag-scheduler-event-loop INFO DAGScheduler: Job 73 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.119 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 73: Stage finished 20/08/03 12:50:47.119 stream execution thread for [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] INFO DAGScheduler: Job 73 finished: testStream$ at LogisticRegressionSuite.scala:39, took 0.093155 s 20/08/03 12:50:47.119 stream execution thread for [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] INFO WriteToDataSourceV2Exec: Data source write support org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@7bd444cb is committing. 20/08/03 12:50:47.122 stream execution thread for [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] INFO WriteToDataSourceV2Exec: Data source write support org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@7bd444cb committed. 20/08/03 12:50:47.134 stream execution thread for [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-9ccc101c-bae5-4c74-9f05-46166fcec276/commits/0 using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-9ccc101c-bae5-4c74-9f05-46166fcec276/commits/.0.d5e96892-e3cf-44ea-bce1-dbe7ab728771.tmp 20/08/03 12:50:47.151 stream execution thread for [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-9ccc101c-bae5-4c74-9f05-46166fcec276/commits/.0.d5e96892-e3cf-44ea-bce1-dbe7ab728771.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-9ccc101c-bae5-4c74-9f05-46166fcec276/commits/0 20/08/03 12:50:47.161 stream execution thread for [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] INFO MicroBatchExecution: Streaming query made progress: { "id" : "0739a101-ca0f-4949-94da-a742f548541b", "runId" : "da58b791-8868-476d-a542-73c733f1113a", "name" : null, "timestamp" : "2020-08-03T19:50:46.792Z", "batchId" : 0, "numInputRows" : 100, "inputRowsPerSecond" : 2173.913043478261, "processedRowsPerSecond" : 277.77777777777777, "durationMs" : { "addBatch" : 205, "getBatch" : 4, "latestOffset" : 0, "queryPlanning" : 66, "triggerExecution" : 360, "walCommit" : 47 }, "stateOperators" : [ ], "sources" : [ { "description" : "MemoryStream[_1#773,_2#774]", "startOffset" : null, "endOffset" : 0, "numInputRows" : 100, "inputRowsPerSecond" : 2173.913043478261, "processedRowsPerSecond" : 277.77777777777777 } ], "sink" : { "description" : "MemorySink", "numOutputRows" : 100 } } 20/08/03 12:50:47.178 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Asked to cancel job group da58b791-8868-476d-a542-73c733f1113a 20/08/03 12:50:47.189 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Asked to cancel job group da58b791-8868-476d-a542-73c733f1113a 20/08/03 12:50:47.190 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Query [id = 0739a101-ca0f-4949-94da-a742f548541b, runId = da58b791-8868-476d-a542-73c733f1113a] was stopped 20/08/03 12:50:47.305 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: StartStream(ProcessingTimeTrigger(0),org.apache.spark.util.SystemClock@48ffe193,Map(),null) 20/08/03 12:50:47.306 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Checkpoint root /home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-ced266f3-c368-45d6-b133-cc012dad2af9 resolved to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-ced266f3-c368-45d6-b133-cc012dad2af9. 20/08/03 12:50:47.314 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-ced266f3-c368-45d6-b133-cc012dad2af9/metadata using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-ced266f3-c368-45d6-b133-cc012dad2af9/.metadata.bb7489b8-d1c5-4445-a0f8-1077da28ca42.tmp 20/08/03 12:50:47.331 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-ced266f3-c368-45d6-b133-cc012dad2af9/.metadata.bb7489b8-d1c5-4445-a0f8-1077da28ca42.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-ced266f3-c368-45d6-b133-cc012dad2af9/metadata 20/08/03 12:50:47.341 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Starting [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a]. Use file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-ced266f3-c368-45d6-b133-cc012dad2af9 to store the query checkpoint. 20/08/03 12:50:47.344 stream execution thread for [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a] INFO MicroBatchExecution: Reading table [org.apache.spark.sql.execution.streaming.MemoryStreamTable@6bf8c398] from DataSourceV2 named 'memory' [org.apache.spark.sql.execution.streaming.MemoryStreamTableProvider$@490dc5fd] 20/08/03 12:50:47.346 stream execution thread for [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a] INFO MicroBatchExecution: Starting new streaming query. 20/08/03 12:50:47.346 stream execution thread for [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a] INFO MicroBatchExecution: Stream started from {} 20/08/03 12:50:47.346 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: AddData to MemoryStream[_1#853,_2#854]: (1.0,[1.1419053154730547]),(0.0,[0.9194079489827879]),(0.0,[-0.9498666368908959]),(1.0,[-1.1069902863993377]),(0.0,[0.2809776380727795]),(1.0,[0.6846227956326554]),(1.0,[-0.8172214073987268]),(0.0,[-1.3966434026780434]),(1.0,[-0.19094451307087512]),(1.0,[1.4862133923906502]),(1.0,[0.8023071496873626]),(0.0,[-0.12151292466549345]),(1.0,[1.4105062239438624]),(0.0,[-0.6402327822135738]),(0.0,[-1.2096444592532913]),(1.0,[0.35375769787202876]),(0.0,[-0.4903496491990076]),(0.0,[0.5507215382743629]),(1.0,[-1.2035510019650835]),(1.0,[0.3210160806416416]),(1.0,[1.5511476388671834]),(1.0,[0.43853028624710505]),(1.0,[0.4815980608245389]),(1.0,[1.5196310789680683]),(0.0,[-0.2768317291873249]),(1.0,[-0.08393897849486337]),(1.0,[1.255833005788796]),(0.0,[-0.3252727938665772]),(0.0,[-0.17329033306108363]),(0.0,[-1.8585851445864527]),(1.0,[1.4238069456328435]),(0.0,[-1.363726024075023]),(0.0,[-1.964666098753878]),(1.0,[-0.9185948439341892]),(0.0,[-2.548887393384806]),(0.0,[-1.6309606578419305]),(1.0,[-0.12200477461989162]),(1.0,[1.289159071801577]),(1.0,[-0.2691388556559934]),(1.0,[0.2574914085090889]),(0.0,[-0.3199143760045327]),(0.0,[-1.7684998592513064]),(1.0,[-0.4834503128592458]),(1.0,[-0.5099904653893699]),(1.0,[1.1166733769661994]),(1.0,[-0.04094720151728288]),(0.0,[-1.1076715169200795]),(1.0,[1.8623214176471945]),(1.0,[1.1457411377091524]),(1.0,[-1.0586772048930921]),(1.0,[1.0725991339400673]),(1.0,[-1.9317441520296659]),(1.0,[0.30102521611534994]),(1.0,[0.2475231582804265]),(1.0,[1.406156849249087]),(0.0,[-1.5202207203569256]),(1.0,[0.2709294126920897]),(1.0,[0.561249284813777]),(0.0,[-0.5298295780368607]),(0.0,[0.5390221914988275]),(1.0,[2.2123402141787243]),(1.0,[-0.6329335687728442]),(0.0,[-1.8831759122084633]),(1.0,[0.3865659853763343]),(1.0,[0.32582927090649455]),(0.0,[-0.9013043195000002]),(1.0,[-0.002680308907617573]),(1.0,[-0.4739592549853249]),(1.0,[-0.5479781547659026]),(1.0,[-0.01910014847196348]),(1.0,[1.6468163882596327]),(0.0,[-1.107062592215791]),(1.0,[0.5938103926672539]),(0.0,[-0.15566462108511642]),(0.0,[0.6632872929286855]),(1.0,[1.226793360688623]),(1.0,[0.8839698437730904]),(0.0,[0.22172454670212935]),(1.0,[0.9197020859698617]),(0.0,[-0.7393758185888677]),(1.0,[0.803517749531419]),(1.0,[-0.2539417447630359]),(1.0,[-0.7638388605060555]),(0.0,[-1.8645567427274516]),(0.0,[-1.861306200027518]),(0.0,[-0.576599881116305]),(1.0,[-0.40899380621224757]),(1.0,[0.24846093761654187]),(1.0,[-0.48091295490277447]),(1.0,[0.44621205735391023]),(1.0,[-0.4465888888803913]),(1.0,[0.045638687865053575]),(1.0,[0.7045663273135641]),(1.0,[-0.2718240183671583]),(1.0,[0.08074877915238832]),(1.0,[1.2590964696340183]),(1.0,[0.7635098382407334]),(1.0,[1.7220810801509723]),(1.0,[0.14595005405372477]),(0.0,[-0.9946630124621867]) 20/08/03 12:50:47.347 stream execution thread for [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a] INFO MicroBatchExecution: Streaming query made progress: { "id" : "7bd4555f-43ea-4d49-8217-faee692f54ee", "runId" : "7aff51d3-7ec7-43e4-b083-77416b45795a", "name" : null, "timestamp" : "2020-08-03T19:50:47.345Z", "batchId" : 0, "numInputRows" : 0, "processedRowsPerSecond" : 0.0, "durationMs" : { "latestOffset" : 0, "triggerExecution" : 1 }, "stateOperators" : [ ], "sources" : [ { "description" : "MemoryStream[_1#853,_2#854]", "startOffset" : null, "endOffset" : null, "numInputRows" : 0, "processedRowsPerSecond" : 0.0 } ], "sink" : { "description" : "MemorySink", "numOutputRows" : -1 } } 20/08/03 12:50:47.349 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: CheckAnswerByFunc 20/08/03 12:50:47.365 stream execution thread for [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a] INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-ced266f3-c368-45d6-b133-cc012dad2af9/offsets/0 using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-ced266f3-c368-45d6-b133-cc012dad2af9/offsets/.0.a64558df-f94a-4ba6-a923-7dacbb8fc1a3.tmp 20/08/03 12:50:47.375 stream execution thread for [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a] INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-ced266f3-c368-45d6-b133-cc012dad2af9/offsets/.0.a64558df-f94a-4ba6-a923-7dacbb8fc1a3.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-ced266f3-c368-45d6-b133-cc012dad2af9/offsets/0 20/08/03 12:50:47.375 stream execution thread for [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a] INFO MicroBatchExecution: Committed offsets for batch 0. Metadata OffsetSeqMetadata(0,1596484247358,Map(spark.sql.streaming.stateStore.providerClass -> org.apache.spark.sql.execution.streaming.state.HDFSBackedStateStoreProvider, spark.sql.streaming.join.stateFormatVersion -> 2, spark.sql.streaming.flatMapGroupsWithState.stateFormatVersion -> 2, spark.sql.streaming.multipleWatermarkPolicy -> min, spark.sql.streaming.aggregation.stateFormatVersion -> 2, spark.sql.shuffle.partitions -> 5)) 20/08/03 12:50:47.420 stream execution thread for [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a] INFO WriteToDataSourceV2Exec: Start processing data source write support: org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@24023aac. The input RDD has 1 partitions. 20/08/03 12:50:47.421 stream execution thread for [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a] INFO SparkContext: Starting job: testStream$ at LogisticRegressionSuite.scala:39 20/08/03 12:50:47.422 dag-scheduler-event-loop INFO DAGScheduler: Got job 74 (testStream$ at LogisticRegressionSuite.scala:39) with 1 output partitions 20/08/03 12:50:47.422 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 74 (testStream$ at LogisticRegressionSuite.scala:39) 20/08/03 12:50:47.422 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.422 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.422 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 74 (MapPartitionsRDD[383] at testStream$ at LogisticRegressionSuite.scala:39), which has no missing parents 20/08/03 12:50:47.424 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_145 stored as values in memory (estimated size 24.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.425 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_145_piece0 stored as bytes in memory (estimated size 10.4 KiB, free 2.1 GiB) 20/08/03 12:50:47.425 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_145_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.4 KiB, free: 2.1 GiB) 20/08/03 12:50:47.425 dag-scheduler-event-loop INFO SparkContext: Created broadcast 145 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.426 dag-scheduler-event-loop INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 74 (MapPartitionsRDD[383] at testStream$ at LogisticRegressionSuite.scala:39) (first 15 tasks are for partitions Vector(0)) 20/08/03 12:50:47.426 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 74.0 with 1 tasks 20/08/03 12:50:47.426 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 74.0 (TID 227, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 18076 bytes) 20/08/03 12:50:47.427 Executor task launch worker for task 227 INFO Executor: Running task 0.0 in stage 74.0 (TID 227) 20/08/03 12:50:47.434 Executor task launch worker for task 227 INFO DataWritingSparkTask: Commit authorized for partition 0 (task 227, attempt 0, stage 74.0) 20/08/03 12:50:47.434 Executor task launch worker for task 227 INFO DataWritingSparkTask: Committed partition 0 (task 227, attempt 0, stage 74.0) 20/08/03 12:50:47.435 Executor task launch worker for task 227 INFO Executor: Finished task 0.0 in stage 74.0 (TID 227). 6472 bytes result sent to driver 20/08/03 12:50:47.435 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 74.0 (TID 227) in 9 ms on amp-jenkins-worker-06.amp (executor driver) (1/1) 20/08/03 12:50:47.436 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 74.0, whose tasks have all completed, from pool 20/08/03 12:50:47.436 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 74 (testStream$ at LogisticRegressionSuite.scala:39) finished in 0.013 s 20/08/03 12:50:47.436 dag-scheduler-event-loop INFO DAGScheduler: Job 74 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.436 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 74: Stage finished 20/08/03 12:50:47.436 stream execution thread for [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a] INFO DAGScheduler: Job 74 finished: testStream$ at LogisticRegressionSuite.scala:39, took 0.015082 s 20/08/03 12:50:47.437 stream execution thread for [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a] INFO WriteToDataSourceV2Exec: Data source write support org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@24023aac is committing. 20/08/03 12:50:47.437 stream execution thread for [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a] INFO WriteToDataSourceV2Exec: Data source write support org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@24023aac committed. 20/08/03 12:50:47.444 stream execution thread for [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a] INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-ced266f3-c368-45d6-b133-cc012dad2af9/commits/0 using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-ced266f3-c368-45d6-b133-cc012dad2af9/commits/.0.a68a6aba-d246-435b-a6ca-2bdead275cc4.tmp 20/08/03 12:50:47.461 stream execution thread for [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a] INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-ced266f3-c368-45d6-b133-cc012dad2af9/commits/.0.a68a6aba-d246-435b-a6ca-2bdead275cc4.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-ced266f3-c368-45d6-b133-cc012dad2af9/commits/0 20/08/03 12:50:47.464 stream execution thread for [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a] INFO MicroBatchExecution: Streaming query made progress: { "id" : "7bd4555f-43ea-4d49-8217-faee692f54ee", "runId" : "7aff51d3-7ec7-43e4-b083-77416b45795a", "name" : null, "timestamp" : "2020-08-03T19:50:47.358Z", "batchId" : 0, "numInputRows" : 100, "inputRowsPerSecond" : 7692.307692307692, "processedRowsPerSecond" : 961.5384615384615, "durationMs" : { "addBatch" : 31, "getBatch" : 0, "latestOffset" : 0, "queryPlanning" : 29, "triggerExecution" : 104, "walCommit" : 17 }, "stateOperators" : [ ], "sources" : [ { "description" : "MemoryStream[_1#853,_2#854]", "startOffset" : null, "endOffset" : 0, "numInputRows" : 100, "inputRowsPerSecond" : 7692.307692307692, "processedRowsPerSecond" : 961.5384615384615 } ], "sink" : { "description" : "MemorySink", "numOutputRows" : 100 } } 20/08/03 12:50:47.467 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Asked to cancel job group 7aff51d3-7ec7-43e4-b083-77416b45795a 20/08/03 12:50:47.469 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Asked to cancel job group 7aff51d3-7ec7-43e4-b083-77416b45795a 20/08/03 12:50:47.469 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Query [id = 7bd4555f-43ea-4d49-8217-faee692f54ee, runId = 7aff51d3-7ec7-43e4-b083-77416b45795a] was stopped 20/08/03 12:50:47.553 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [d2b78bff] Stage class: LogisticRegression 20/08/03 12:50:47.553 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [d2b78bff] Stage uid: logreg_2bcbcc838498 20/08/03 12:50:47.577 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [d2b78bff] training: numPartitions=4 storageLevel=StorageLevel(1 replicas) 20/08/03 12:50:47.577 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [d2b78bff] {} 20/08/03 12:50:47.585 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at LogisticRegression.scala:499 20/08/03 12:50:47.585 dag-scheduler-event-loop INFO DAGScheduler: Got job 75 (treeAggregate at LogisticRegression.scala:499) with 4 output partitions 20/08/03 12:50:47.586 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 75 (treeAggregate at LogisticRegression.scala:499) 20/08/03 12:50:47.586 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.587 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.587 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 75 (MapPartitionsRDD[395] at treeAggregate at LogisticRegression.scala:499), which has no missing parents 20/08/03 12:50:47.591 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_146 stored as values in memory (estimated size 27.5 KiB, free 2.1 GiB) 20/08/03 12:50:47.593 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_146_piece0 stored as bytes in memory (estimated size 10.8 KiB, free 2.1 GiB) 20/08/03 12:50:47.593 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_146_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.8 KiB, free: 2.1 GiB) 20/08/03 12:50:47.593 dag-scheduler-event-loop INFO SparkContext: Created broadcast 146 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.594 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 75 (MapPartitionsRDD[395] at treeAggregate at LogisticRegression.scala:499) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:47.594 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 75.0 with 4 tasks 20/08/03 12:50:47.594 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 75.0 (TID 228, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.595 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 75.0 (TID 229, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.595 Executor task launch worker for task 229 INFO Executor: Running task 1.0 in stage 75.0 (TID 229) 20/08/03 12:50:47.595 Executor task launch worker for task 228 INFO Executor: Running task 0.0 in stage 75.0 (TID 228) 20/08/03 12:50:47.603 Executor task launch worker for task 228 INFO MemoryStore: Block rdd_389_0 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:47.603 Executor task launch worker for task 229 INFO MemoryStore: Block rdd_389_1 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:47.603 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_389_0 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:47.603 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_389_1 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:47.604 Executor task launch worker for task 229 INFO Executor: Finished task 1.0 in stage 75.0 (TID 229). 2639 bytes result sent to driver 20/08/03 12:50:47.604 Executor task launch worker for task 228 INFO Executor: Finished task 0.0 in stage 75.0 (TID 228). 2673 bytes result sent to driver 20/08/03 12:50:47.605 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 75.0 (TID 230, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.605 Executor task launch worker for task 230 INFO Executor: Running task 2.0 in stage 75.0 (TID 230) 20/08/03 12:50:47.606 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 75.0 (TID 231, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.606 Executor task launch worker for task 231 INFO Executor: Running task 3.0 in stage 75.0 (TID 231) 20/08/03 12:50:47.606 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 75.0 (TID 229) in 11 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:47.606 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 75.0 (TID 228) in 12 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:47.610 Executor task launch worker for task 230 INFO MemoryStore: Block rdd_389_2 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:47.610 Executor task launch worker for task 231 INFO MemoryStore: Block rdd_389_3 stored as values in memory (estimated size 2.1 KiB, free 2.1 GiB) 20/08/03 12:50:47.610 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_389_2 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:47.610 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added rdd_389_3 in memory on amp-jenkins-worker-06.amp:34426 (size: 2.1 KiB, free: 2.1 GiB) 20/08/03 12:50:47.611 Executor task launch worker for task 231 INFO Executor: Finished task 3.0 in stage 75.0 (TID 231). 2639 bytes result sent to driver 20/08/03 12:50:47.611 Executor task launch worker for task 230 INFO Executor: Finished task 2.0 in stage 75.0 (TID 230). 2673 bytes result sent to driver 20/08/03 12:50:47.611 task-result-getter-2 INFO TaskSetManager: Finished task 3.0 in stage 75.0 (TID 231) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:47.611 task-result-getter-3 INFO TaskSetManager: Finished task 2.0 in stage 75.0 (TID 230) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:47.611 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 75.0, whose tasks have all completed, from pool 20/08/03 12:50:47.612 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 75 (treeAggregate at LogisticRegression.scala:499) finished in 0.025 s 20/08/03 12:50:47.612 dag-scheduler-event-loop INFO DAGScheduler: Job 75 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.612 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 75: Stage finished 20/08/03 12:50:47.612 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 75 finished: treeAggregate at LogisticRegression.scala:499, took 0.027178 s 20/08/03 12:50:47.613 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [d2b78bff] {"numExamples":100} 20/08/03 12:50:47.613 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [d2b78bff] {"lowestLabelWeight":"3.0"} 20/08/03 12:50:47.613 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [d2b78bff] {"highestLabelWeight":"67.0"} 20/08/03 12:50:47.613 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [d2b78bff] {"sumOfWeights":100.0} 20/08/03 12:50:47.613 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [d2b78bff] {"numClasses":3} 20/08/03 12:50:47.613 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [d2b78bff] {"numFeatures":2} 20/08/03 12:50:47.614 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_147 stored as values in memory (estimated size 56.0 B, free 2.1 GiB) 20/08/03 12:50:47.615 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_147_piece0 stored as bytes in memory (estimated size 85.0 B, free 2.1 GiB) 20/08/03 12:50:47.615 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_147_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 85.0 B, free: 2.1 GiB) 20/08/03 12:50:47.616 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 147 from broadcast at LogisticRegression.scala:593 20/08/03 12:50:47.616 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_148 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:47.618 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_148_piece0 stored as bytes in memory (estimated size 171.0 B, free 2.1 GiB) 20/08/03 12:50:47.618 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_148_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 171.0 B, free: 2.1 GiB) 20/08/03 12:50:47.618 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 148 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:47.628 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:47.628 dag-scheduler-event-loop INFO DAGScheduler: Got job 76 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:47.628 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 76 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:47.628 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.629 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.629 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 76 (MapPartitionsRDD[396] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:47.631 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_149 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.632 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_149_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:47.632 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_149_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:47.632 dag-scheduler-event-loop INFO SparkContext: Created broadcast 149 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.633 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 76 (MapPartitionsRDD[396] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:47.633 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 76.0 with 4 tasks 20/08/03 12:50:47.634 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 76.0 (TID 232, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.634 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 76.0 (TID 233, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.634 Executor task launch worker for task 232 INFO Executor: Running task 0.0 in stage 76.0 (TID 232) 20/08/03 12:50:47.634 Executor task launch worker for task 233 INFO Executor: Running task 1.0 in stage 76.0 (TID 233) 20/08/03 12:50:47.637 Executor task launch worker for task 233 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:47.637 Executor task launch worker for task 232 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:47.638 Executor task launch worker for task 232 INFO Executor: Finished task 0.0 in stage 76.0 (TID 232). 2339 bytes result sent to driver 20/08/03 12:50:47.638 Executor task launch worker for task 233 INFO Executor: Finished task 1.0 in stage 76.0 (TID 233). 2296 bytes result sent to driver 20/08/03 12:50:47.638 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 76.0 (TID 234, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.638 Executor task launch worker for task 234 INFO Executor: Running task 2.0 in stage 76.0 (TID 234) 20/08/03 12:50:47.638 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 76.0 (TID 235, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.638 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 76.0 (TID 232) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:47.639 Executor task launch worker for task 235 INFO Executor: Running task 3.0 in stage 76.0 (TID 235) 20/08/03 12:50:47.639 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 76.0 (TID 233) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:47.640 Executor task launch worker for task 234 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:47.640 Executor task launch worker for task 235 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:47.640 Executor task launch worker for task 234 INFO Executor: Finished task 2.0 in stage 76.0 (TID 234). 2296 bytes result sent to driver 20/08/03 12:50:47.641 Executor task launch worker for task 235 INFO Executor: Finished task 3.0 in stage 76.0 (TID 235). 2296 bytes result sent to driver 20/08/03 12:50:47.641 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 76.0 (TID 234) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:47.641 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 76.0 (TID 235) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:47.641 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 76.0, whose tasks have all completed, from pool 20/08/03 12:50:47.641 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 76 (treeAggregate at RDDLossFunction.scala:61) finished in 0.012 s 20/08/03 12:50:47.642 dag-scheduler-event-loop INFO DAGScheduler: Job 76 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.642 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 76: Stage finished 20/08/03 12:50:47.642 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 76 finished: treeAggregate at RDDLossFunction.scala:61, took 0.014099 s 20/08/03 12:50:47.642 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(148) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:47.643 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_148_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 171.0 B, free: 2.1 GiB) 20/08/03 12:50:47.644 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_150 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:47.645 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_150_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:47.645 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_150_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.646 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 150 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:47.655 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:47.655 dag-scheduler-event-loop INFO DAGScheduler: Got job 77 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:47.655 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 77 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:47.656 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.656 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.656 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 77 (MapPartitionsRDD[397] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:47.658 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_151 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.659 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_151_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:47.659 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_151_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:47.660 dag-scheduler-event-loop INFO SparkContext: Created broadcast 151 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.660 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 77 (MapPartitionsRDD[397] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:47.660 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 77.0 with 4 tasks 20/08/03 12:50:47.661 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 77.0 (TID 236, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.661 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 77.0 (TID 237, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.661 Executor task launch worker for task 237 INFO Executor: Running task 1.0 in stage 77.0 (TID 237) 20/08/03 12:50:47.661 Executor task launch worker for task 236 INFO Executor: Running task 0.0 in stage 77.0 (TID 236) 20/08/03 12:50:47.664 Executor task launch worker for task 237 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:47.664 Executor task launch worker for task 236 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:47.664 Executor task launch worker for task 236 INFO Executor: Finished task 0.0 in stage 77.0 (TID 236). 2339 bytes result sent to driver 20/08/03 12:50:47.664 Executor task launch worker for task 237 INFO Executor: Finished task 1.0 in stage 77.0 (TID 237). 2296 bytes result sent to driver 20/08/03 12:50:47.665 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 77.0 (TID 238, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.665 Executor task launch worker for task 238 INFO Executor: Running task 2.0 in stage 77.0 (TID 238) 20/08/03 12:50:47.665 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 77.0 (TID 239, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.665 Executor task launch worker for task 239 INFO Executor: Running task 3.0 in stage 77.0 (TID 239) 20/08/03 12:50:47.665 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 77.0 (TID 236) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:47.665 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 77.0 (TID 237) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:47.666 Executor task launch worker for task 238 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:47.667 Executor task launch worker for task 239 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:47.667 Executor task launch worker for task 238 INFO Executor: Finished task 2.0 in stage 77.0 (TID 238). 2296 bytes result sent to driver 20/08/03 12:50:47.667 Executor task launch worker for task 239 INFO Executor: Finished task 3.0 in stage 77.0 (TID 239). 2296 bytes result sent to driver 20/08/03 12:50:47.667 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 77.0 (TID 238) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:47.668 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 77.0 (TID 239) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:47.668 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 77.0, whose tasks have all completed, from pool 20/08/03 12:50:47.668 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 77 (treeAggregate at RDDLossFunction.scala:61) finished in 0.011 s 20/08/03 12:50:47.668 dag-scheduler-event-loop INFO DAGScheduler: Job 77 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.668 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 77: Stage finished 20/08/03 12:50:47.668 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 77 finished: treeAggregate at RDDLossFunction.scala:61, took 0.013221 s 20/08/03 12:50:47.669 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(150) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:47.669 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_150_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.670 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_152 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:47.671 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_152_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:47.671 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_152_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.671 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 152 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:47.678 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:47.678 dag-scheduler-event-loop INFO DAGScheduler: Got job 78 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:47.678 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 78 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:47.678 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.678 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.679 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 78 (MapPartitionsRDD[398] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:47.680 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_153 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.681 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_153_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:47.681 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_153_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:47.682 dag-scheduler-event-loop INFO SparkContext: Created broadcast 153 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.682 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 78 (MapPartitionsRDD[398] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:47.682 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 78.0 with 4 tasks 20/08/03 12:50:47.683 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 78.0 (TID 240, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.683 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 78.0 (TID 241, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.683 Executor task launch worker for task 240 INFO Executor: Running task 0.0 in stage 78.0 (TID 240) 20/08/03 12:50:47.683 Executor task launch worker for task 241 INFO Executor: Running task 1.0 in stage 78.0 (TID 241) 20/08/03 12:50:47.686 Executor task launch worker for task 240 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:47.686 Executor task launch worker for task 241 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:47.686 Executor task launch worker for task 240 INFO Executor: Finished task 0.0 in stage 78.0 (TID 240). 2339 bytes result sent to driver 20/08/03 12:50:47.686 Executor task launch worker for task 241 INFO Executor: Finished task 1.0 in stage 78.0 (TID 241). 2296 bytes result sent to driver 20/08/03 12:50:47.687 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 78.0 (TID 242, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.687 Executor task launch worker for task 242 INFO Executor: Running task 2.0 in stage 78.0 (TID 242) 20/08/03 12:50:47.687 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 78.0 (TID 243, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.687 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 78.0 (TID 240) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:47.687 Executor task launch worker for task 243 INFO Executor: Running task 3.0 in stage 78.0 (TID 243) 20/08/03 12:50:47.687 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 78.0 (TID 241) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:47.689 Executor task launch worker for task 242 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:47.689 Executor task launch worker for task 243 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:47.689 Executor task launch worker for task 242 INFO Executor: Finished task 2.0 in stage 78.0 (TID 242). 2296 bytes result sent to driver 20/08/03 12:50:47.689 Executor task launch worker for task 243 INFO Executor: Finished task 3.0 in stage 78.0 (TID 243). 2296 bytes result sent to driver 20/08/03 12:50:47.689 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 78.0 (TID 242) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:47.689 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 78.0 (TID 243) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:47.690 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 78.0, whose tasks have all completed, from pool 20/08/03 12:50:47.690 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 78 (treeAggregate at RDDLossFunction.scala:61) finished in 0.011 s 20/08/03 12:50:47.690 dag-scheduler-event-loop INFO DAGScheduler: Job 78 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.690 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 78: Stage finished 20/08/03 12:50:47.690 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 78 finished: treeAggregate at RDDLossFunction.scala:61, took 0.012366 s 20/08/03 12:50:47.691 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(152) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:47.691 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO StrongWolfeLineSearch: Line search t: 0.5508985649472954 fval: 0.7342215028617264 rhs: 0.7358720183663033 cdd: 0.02405787096453525 20/08/03 12:50:47.691 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 0.5509 20/08/03 12:50:47.691 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_152_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.691 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.734222 (rel: 0.00225) 0.789814 20/08/03 12:50:47.692 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_154 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:47.693 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_154_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:47.693 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_154_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.694 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 154 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:47.702 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:47.702 dag-scheduler-event-loop INFO DAGScheduler: Got job 79 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:47.702 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 79 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:47.702 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.703 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.703 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 79 (MapPartitionsRDD[399] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:47.705 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_155 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.706 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_155_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:47.706 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_155_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:47.706 dag-scheduler-event-loop INFO SparkContext: Created broadcast 155 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.707 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 79 (MapPartitionsRDD[399] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:47.707 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 79.0 with 4 tasks 20/08/03 12:50:47.708 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 79.0 (TID 244, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.708 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 79.0 (TID 245, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.708 Executor task launch worker for task 244 INFO Executor: Running task 0.0 in stage 79.0 (TID 244) 20/08/03 12:50:47.708 Executor task launch worker for task 245 INFO Executor: Running task 1.0 in stage 79.0 (TID 245) 20/08/03 12:50:47.711 Executor task launch worker for task 244 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:47.711 Executor task launch worker for task 245 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:47.712 Executor task launch worker for task 244 INFO Executor: Finished task 0.0 in stage 79.0 (TID 244). 2296 bytes result sent to driver 20/08/03 12:50:47.712 Executor task launch worker for task 245 INFO Executor: Finished task 1.0 in stage 79.0 (TID 245). 2339 bytes result sent to driver 20/08/03 12:50:47.712 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 79.0 (TID 246, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.712 Executor task launch worker for task 246 INFO Executor: Running task 2.0 in stage 79.0 (TID 246) 20/08/03 12:50:47.713 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 79.0 (TID 247, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.713 Executor task launch worker for task 247 INFO Executor: Running task 3.0 in stage 79.0 (TID 247) 20/08/03 12:50:47.713 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 79.0 (TID 244) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:47.713 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 79.0 (TID 245) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:47.715 Executor task launch worker for task 246 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:47.715 Executor task launch worker for task 247 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:47.715 Executor task launch worker for task 246 INFO Executor: Finished task 2.0 in stage 79.0 (TID 246). 2296 bytes result sent to driver 20/08/03 12:50:47.716 Executor task launch worker for task 247 INFO Executor: Finished task 3.0 in stage 79.0 (TID 247). 2296 bytes result sent to driver 20/08/03 12:50:47.716 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 79.0 (TID 246) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:47.716 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 79.0 (TID 247) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:47.716 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 79.0, whose tasks have all completed, from pool 20/08/03 12:50:47.716 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 79 (treeAggregate at RDDLossFunction.scala:61) finished in 0.013 s 20/08/03 12:50:47.717 dag-scheduler-event-loop INFO DAGScheduler: Job 79 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.717 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 79: Stage finished 20/08/03 12:50:47.717 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 79 finished: treeAggregate at RDDLossFunction.scala:61, took 0.014955 s 20/08/03 12:50:47.718 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(154) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:47.718 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:47.718 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.724769 (rel: 0.0129) 0.234619 20/08/03 12:50:47.719 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_156 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:47.720 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_154_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.721 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_156_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:47.721 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_156_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.721 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 156 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:47.732 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:47.732 dag-scheduler-event-loop INFO DAGScheduler: Got job 80 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:47.733 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 80 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:47.733 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.733 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.734 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 80 (MapPartitionsRDD[400] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:47.736 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_157 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.737 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_157_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:47.737 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_157_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:47.738 dag-scheduler-event-loop INFO SparkContext: Created broadcast 157 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.738 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 80 (MapPartitionsRDD[400] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:47.738 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 80.0 with 4 tasks 20/08/03 12:50:47.739 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 80.0 (TID 248, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.739 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 80.0 (TID 249, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.739 Executor task launch worker for task 248 INFO Executor: Running task 0.0 in stage 80.0 (TID 248) 20/08/03 12:50:47.739 Executor task launch worker for task 249 INFO Executor: Running task 1.0 in stage 80.0 (TID 249) 20/08/03 12:50:47.743 Executor task launch worker for task 249 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:47.743 Executor task launch worker for task 248 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:47.743 Executor task launch worker for task 249 INFO Executor: Finished task 1.0 in stage 80.0 (TID 249). 2339 bytes result sent to driver 20/08/03 12:50:47.743 Executor task launch worker for task 248 INFO Executor: Finished task 0.0 in stage 80.0 (TID 248). 2296 bytes result sent to driver 20/08/03 12:50:47.744 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 80.0 (TID 250, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.744 Executor task launch worker for task 250 INFO Executor: Running task 2.0 in stage 80.0 (TID 250) 20/08/03 12:50:47.744 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 80.0 (TID 251, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.744 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 80.0 (TID 248) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:47.745 Executor task launch worker for task 251 INFO Executor: Running task 3.0 in stage 80.0 (TID 251) 20/08/03 12:50:47.745 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 80.0 (TID 249) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:47.746 Executor task launch worker for task 250 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:47.747 Executor task launch worker for task 251 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:47.747 Executor task launch worker for task 250 INFO Executor: Finished task 2.0 in stage 80.0 (TID 250). 2296 bytes result sent to driver 20/08/03 12:50:47.747 Executor task launch worker for task 251 INFO Executor: Finished task 3.0 in stage 80.0 (TID 251). 2296 bytes result sent to driver 20/08/03 12:50:47.747 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 80.0 (TID 250) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:47.747 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 80.0 (TID 251) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:47.748 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 80.0, whose tasks have all completed, from pool 20/08/03 12:50:47.748 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 80 (treeAggregate at RDDLossFunction.scala:61) finished in 0.014 s 20/08/03 12:50:47.748 dag-scheduler-event-loop INFO DAGScheduler: Job 80 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.748 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 80: Stage finished 20/08/03 12:50:47.748 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 80 finished: treeAggregate at RDDLossFunction.scala:61, took 0.016560 s 20/08/03 12:50:47.749 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(156) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:47.750 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:47.750 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_156_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.750 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.722474 (rel: 0.00317) 0.175893 20/08/03 12:50:47.751 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_158 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:47.753 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_158_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:47.753 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_158_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.753 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 158 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:47.764 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:47.764 dag-scheduler-event-loop INFO DAGScheduler: Got job 81 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:47.765 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 81 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:47.765 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.765 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.765 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 81 (MapPartitionsRDD[401] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:47.767 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_159 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.769 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_159_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:47.770 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_159_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:47.770 dag-scheduler-event-loop INFO SparkContext: Created broadcast 159 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.771 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 81 (MapPartitionsRDD[401] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:47.771 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 81.0 with 4 tasks 20/08/03 12:50:47.772 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 81.0 (TID 252, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.773 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 81.0 (TID 253, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.773 Executor task launch worker for task 252 INFO Executor: Running task 0.0 in stage 81.0 (TID 252) 20/08/03 12:50:47.773 Executor task launch worker for task 253 INFO Executor: Running task 1.0 in stage 81.0 (TID 253) 20/08/03 12:50:47.776 Executor task launch worker for task 252 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:47.776 Executor task launch worker for task 253 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:47.776 Executor task launch worker for task 253 INFO Executor: Finished task 1.0 in stage 81.0 (TID 253). 2296 bytes result sent to driver 20/08/03 12:50:47.776 Executor task launch worker for task 252 INFO Executor: Finished task 0.0 in stage 81.0 (TID 252). 2296 bytes result sent to driver 20/08/03 12:50:47.777 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 81.0 (TID 254, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.777 Executor task launch worker for task 254 INFO Executor: Running task 2.0 in stage 81.0 (TID 254) 20/08/03 12:50:47.777 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 81.0 (TID 255, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.778 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 81.0 (TID 253) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:47.778 Executor task launch worker for task 255 INFO Executor: Running task 3.0 in stage 81.0 (TID 255) 20/08/03 12:50:47.778 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 81.0 (TID 252) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:47.779 Executor task launch worker for task 254 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:47.779 Executor task launch worker for task 255 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:47.780 Executor task launch worker for task 255 INFO Executor: Finished task 3.0 in stage 81.0 (TID 255). 2296 bytes result sent to driver 20/08/03 12:50:47.780 Executor task launch worker for task 254 INFO Executor: Finished task 2.0 in stage 81.0 (TID 254). 2296 bytes result sent to driver 20/08/03 12:50:47.780 task-result-getter-2 INFO TaskSetManager: Finished task 3.0 in stage 81.0 (TID 255) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:47.780 task-result-getter-3 INFO TaskSetManager: Finished task 2.0 in stage 81.0 (TID 254) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:47.781 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 81.0, whose tasks have all completed, from pool 20/08/03 12:50:47.781 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 81 (treeAggregate at RDDLossFunction.scala:61) finished in 0.015 s 20/08/03 12:50:47.781 dag-scheduler-event-loop INFO DAGScheduler: Job 81 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.781 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 81: Stage finished 20/08/03 12:50:47.781 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 81 finished: treeAggregate at RDDLossFunction.scala:61, took 0.017449 s 20/08/03 12:50:47.782 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(158) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:47.782 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:47.782 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.720299 (rel: 0.00301) 0.232261 20/08/03 12:50:47.783 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_158_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.783 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_160 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:47.785 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_160_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:47.785 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_160_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.785 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 160 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:47.794 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:47.795 dag-scheduler-event-loop INFO DAGScheduler: Got job 82 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:47.795 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 82 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:47.795 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.795 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.795 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 82 (MapPartitionsRDD[402] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:47.797 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_161 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.799 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_161_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:47.799 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_161_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:47.799 dag-scheduler-event-loop INFO SparkContext: Created broadcast 161 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.800 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 82 (MapPartitionsRDD[402] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:47.800 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 82.0 with 4 tasks 20/08/03 12:50:47.801 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 82.0 (TID 256, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.801 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 82.0 (TID 257, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.801 Executor task launch worker for task 256 INFO Executor: Running task 0.0 in stage 82.0 (TID 256) 20/08/03 12:50:47.801 Executor task launch worker for task 257 INFO Executor: Running task 1.0 in stage 82.0 (TID 257) 20/08/03 12:50:47.804 Executor task launch worker for task 256 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:47.805 Executor task launch worker for task 256 INFO Executor: Finished task 0.0 in stage 82.0 (TID 256). 2339 bytes result sent to driver 20/08/03 12:50:47.805 Executor task launch worker for task 257 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:47.806 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 82.0 (TID 258, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.806 Executor task launch worker for task 257 INFO Executor: Finished task 1.0 in stage 82.0 (TID 257). 2339 bytes result sent to driver 20/08/03 12:50:47.806 Executor task launch worker for task 258 INFO Executor: Running task 2.0 in stage 82.0 (TID 258) 20/08/03 12:50:47.806 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 82.0 (TID 259, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.806 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 82.0 (TID 256) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:47.806 Executor task launch worker for task 259 INFO Executor: Running task 3.0 in stage 82.0 (TID 259) 20/08/03 12:50:47.806 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 82.0 (TID 257) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:47.808 Executor task launch worker for task 258 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:47.808 Executor task launch worker for task 259 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:47.808 Executor task launch worker for task 258 INFO Executor: Finished task 2.0 in stage 82.0 (TID 258). 2296 bytes result sent to driver 20/08/03 12:50:47.808 Executor task launch worker for task 259 INFO Executor: Finished task 3.0 in stage 82.0 (TID 259). 2296 bytes result sent to driver 20/08/03 12:50:47.808 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 82.0 (TID 258) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:47.808 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 82.0 (TID 259) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:47.809 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 82.0, whose tasks have all completed, from pool 20/08/03 12:50:47.809 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 82 (treeAggregate at RDDLossFunction.scala:61) finished in 0.013 s 20/08/03 12:50:47.809 dag-scheduler-event-loop INFO DAGScheduler: Job 82 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.809 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 82: Stage finished 20/08/03 12:50:47.809 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 82 finished: treeAggregate at RDDLossFunction.scala:61, took 0.014871 s 20/08/03 12:50:47.810 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(160) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:47.810 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:47.810 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.713961 (rel: 0.00880) 0.296041 20/08/03 12:50:47.810 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_160_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.811 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_162 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:47.812 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_162_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:47.812 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_162_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.813 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 162 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:47.819 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:47.820 dag-scheduler-event-loop INFO DAGScheduler: Got job 83 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:47.820 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 83 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:47.820 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.820 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.821 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 83 (MapPartitionsRDD[403] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:47.822 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_163 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.823 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_163_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:47.823 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_163_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:47.824 dag-scheduler-event-loop INFO SparkContext: Created broadcast 163 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.824 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 83 (MapPartitionsRDD[403] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:47.824 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 83.0 with 4 tasks 20/08/03 12:50:47.825 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 83.0 (TID 260, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.825 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 83.0 (TID 261, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.825 Executor task launch worker for task 260 INFO Executor: Running task 0.0 in stage 83.0 (TID 260) 20/08/03 12:50:47.825 Executor task launch worker for task 261 INFO Executor: Running task 1.0 in stage 83.0 (TID 261) 20/08/03 12:50:47.828 Executor task launch worker for task 260 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:47.828 Executor task launch worker for task 261 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:47.829 Executor task launch worker for task 260 INFO Executor: Finished task 0.0 in stage 83.0 (TID 260). 2339 bytes result sent to driver 20/08/03 12:50:47.829 Executor task launch worker for task 261 INFO Executor: Finished task 1.0 in stage 83.0 (TID 261). 2339 bytes result sent to driver 20/08/03 12:50:47.829 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 83.0 (TID 262, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.830 Executor task launch worker for task 262 INFO Executor: Running task 2.0 in stage 83.0 (TID 262) 20/08/03 12:50:47.830 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 83.0 (TID 263, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.830 Executor task launch worker for task 263 INFO Executor: Running task 3.0 in stage 83.0 (TID 263) 20/08/03 12:50:47.830 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 83.0 (TID 260) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:47.830 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 83.0 (TID 261) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:47.831 Executor task launch worker for task 262 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:47.831 Executor task launch worker for task 263 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:47.832 Executor task launch worker for task 262 INFO Executor: Finished task 2.0 in stage 83.0 (TID 262). 2296 bytes result sent to driver 20/08/03 12:50:47.832 Executor task launch worker for task 263 INFO Executor: Finished task 3.0 in stage 83.0 (TID 263). 2296 bytes result sent to driver 20/08/03 12:50:47.832 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 83.0 (TID 262) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:47.832 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 83.0 (TID 263) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:47.832 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 83.0, whose tasks have all completed, from pool 20/08/03 12:50:47.833 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 83 (treeAggregate at RDDLossFunction.scala:61) finished in 0.011 s 20/08/03 12:50:47.833 dag-scheduler-event-loop INFO DAGScheduler: Job 83 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.833 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 83: Stage finished 20/08/03 12:50:47.833 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 83 finished: treeAggregate at RDDLossFunction.scala:61, took 0.013440 s 20/08/03 12:50:47.833 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(162) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:47.834 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:47.834 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.707818 (rel: 0.00860) 0.240813 20/08/03 12:50:47.834 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_162_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.835 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_164 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:47.836 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_164_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:47.836 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_164_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.836 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 164 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:47.843 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:47.844 dag-scheduler-event-loop INFO DAGScheduler: Got job 84 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:47.844 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 84 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:47.844 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.844 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.844 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 84 (MapPartitionsRDD[404] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:47.846 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_165 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.847 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_165_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:47.847 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_165_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:47.847 dag-scheduler-event-loop INFO SparkContext: Created broadcast 165 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.848 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 84 (MapPartitionsRDD[404] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:47.848 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 84.0 with 4 tasks 20/08/03 12:50:47.849 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 84.0 (TID 264, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.849 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 84.0 (TID 265, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.849 Executor task launch worker for task 264 INFO Executor: Running task 0.0 in stage 84.0 (TID 264) 20/08/03 12:50:47.849 Executor task launch worker for task 265 INFO Executor: Running task 1.0 in stage 84.0 (TID 265) 20/08/03 12:50:47.852 Executor task launch worker for task 264 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:47.852 Executor task launch worker for task 265 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:47.853 Executor task launch worker for task 264 INFO Executor: Finished task 0.0 in stage 84.0 (TID 264). 2339 bytes result sent to driver 20/08/03 12:50:47.853 Executor task launch worker for task 265 INFO Executor: Finished task 1.0 in stage 84.0 (TID 265). 2296 bytes result sent to driver 20/08/03 12:50:47.853 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 84.0 (TID 266, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.853 Executor task launch worker for task 266 INFO Executor: Running task 2.0 in stage 84.0 (TID 266) 20/08/03 12:50:47.853 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 84.0 (TID 267, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.853 Executor task launch worker for task 267 INFO Executor: Running task 3.0 in stage 84.0 (TID 267) 20/08/03 12:50:47.853 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 84.0 (TID 264) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:47.853 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 84.0 (TID 265) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:47.855 Executor task launch worker for task 266 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:47.855 Executor task launch worker for task 267 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:47.855 Executor task launch worker for task 266 INFO Executor: Finished task 2.0 in stage 84.0 (TID 266). 2296 bytes result sent to driver 20/08/03 12:50:47.855 Executor task launch worker for task 267 INFO Executor: Finished task 3.0 in stage 84.0 (TID 267). 2296 bytes result sent to driver 20/08/03 12:50:47.855 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 84.0 (TID 266) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:47.856 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 84.0 (TID 267) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:47.856 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 84.0, whose tasks have all completed, from pool 20/08/03 12:50:47.856 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 84 (treeAggregate at RDDLossFunction.scala:61) finished in 0.011 s 20/08/03 12:50:47.856 dag-scheduler-event-loop INFO DAGScheduler: Job 84 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.856 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 84: Stage finished 20/08/03 12:50:47.856 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 84 finished: treeAggregate at RDDLossFunction.scala:61, took 0.012918 s 20/08/03 12:50:47.857 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(164) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:47.857 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:47.857 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.707706 (rel: 0.000159) 0.143614 20/08/03 12:50:47.857 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_164_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.858 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_166 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:47.859 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_166_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:47.859 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_166_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.860 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 166 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:47.866 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:47.867 dag-scheduler-event-loop INFO DAGScheduler: Got job 85 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:47.867 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 85 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:47.867 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.867 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.867 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 85 (MapPartitionsRDD[405] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:47.869 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_167 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.870 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_167_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:47.870 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_167_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:47.871 dag-scheduler-event-loop INFO SparkContext: Created broadcast 167 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.871 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 85 (MapPartitionsRDD[405] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:47.871 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 85.0 with 4 tasks 20/08/03 12:50:47.872 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 85.0 (TID 268, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.872 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 85.0 (TID 269, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.872 Executor task launch worker for task 269 INFO Executor: Running task 1.0 in stage 85.0 (TID 269) 20/08/03 12:50:47.872 Executor task launch worker for task 268 INFO Executor: Running task 0.0 in stage 85.0 (TID 268) 20/08/03 12:50:47.876 Executor task launch worker for task 268 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:47.876 Executor task launch worker for task 269 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:47.877 Executor task launch worker for task 269 INFO Executor: Finished task 1.0 in stage 85.0 (TID 269). 2296 bytes result sent to driver 20/08/03 12:50:47.877 Executor task launch worker for task 268 INFO Executor: Finished task 0.0 in stage 85.0 (TID 268). 2339 bytes result sent to driver 20/08/03 12:50:47.878 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 85.0 (TID 270, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.878 Executor task launch worker for task 270 INFO Executor: Running task 2.0 in stage 85.0 (TID 270) 20/08/03 12:50:47.878 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 85.0 (TID 271, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.878 Executor task launch worker for task 271 INFO Executor: Running task 3.0 in stage 85.0 (TID 271) 20/08/03 12:50:47.878 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 85.0 (TID 269) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:47.878 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 85.0 (TID 268) in 7 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:47.880 Executor task launch worker for task 270 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:47.880 Executor task launch worker for task 270 INFO Executor: Finished task 2.0 in stage 85.0 (TID 270). 2296 bytes result sent to driver 20/08/03 12:50:47.880 Executor task launch worker for task 271 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:47.880 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 85.0 (TID 270) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:47.881 Executor task launch worker for task 271 INFO Executor: Finished task 3.0 in stage 85.0 (TID 271). 2296 bytes result sent to driver 20/08/03 12:50:47.881 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 85.0 (TID 271) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:47.881 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 85.0, whose tasks have all completed, from pool 20/08/03 12:50:47.881 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 85 (treeAggregate at RDDLossFunction.scala:61) finished in 0.013 s 20/08/03 12:50:47.881 dag-scheduler-event-loop INFO DAGScheduler: Job 85 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.882 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 85: Stage finished 20/08/03 12:50:47.882 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 85 finished: treeAggregate at RDDLossFunction.scala:61, took 0.015162 s 20/08/03 12:50:47.882 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(166) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:47.883 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:47.883 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_166_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.883 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.706803 (rel: 0.00128) 0.0179753 20/08/03 12:50:47.884 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_168 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:47.885 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_168_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:47.885 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_168_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.885 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 168 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:47.892 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:47.892 dag-scheduler-event-loop INFO DAGScheduler: Got job 86 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:47.892 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 86 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:47.892 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.892 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.893 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 86 (MapPartitionsRDD[406] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:47.894 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_169 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.895 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_169_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:47.895 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_169_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:47.895 dag-scheduler-event-loop INFO SparkContext: Created broadcast 169 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.896 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 86 (MapPartitionsRDD[406] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:47.896 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 86.0 with 4 tasks 20/08/03 12:50:47.896 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 86.0 (TID 272, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.897 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 86.0 (TID 273, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.897 Executor task launch worker for task 272 INFO Executor: Running task 0.0 in stage 86.0 (TID 272) 20/08/03 12:50:47.897 Executor task launch worker for task 273 INFO Executor: Running task 1.0 in stage 86.0 (TID 273) 20/08/03 12:50:47.899 Executor task launch worker for task 272 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:47.899 Executor task launch worker for task 273 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:47.899 Executor task launch worker for task 272 INFO Executor: Finished task 0.0 in stage 86.0 (TID 272). 2296 bytes result sent to driver 20/08/03 12:50:47.899 Executor task launch worker for task 273 INFO Executor: Finished task 1.0 in stage 86.0 (TID 273). 2296 bytes result sent to driver 20/08/03 12:50:47.899 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 86.0 (TID 274, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.899 Executor task launch worker for task 274 INFO Executor: Running task 2.0 in stage 86.0 (TID 274) 20/08/03 12:50:47.900 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 86.0 (TID 275, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.900 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 86.0 (TID 272) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:47.900 Executor task launch worker for task 275 INFO Executor: Running task 3.0 in stage 86.0 (TID 275) 20/08/03 12:50:47.900 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 86.0 (TID 273) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:47.901 Executor task launch worker for task 274 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:47.901 Executor task launch worker for task 274 INFO Executor: Finished task 2.0 in stage 86.0 (TID 274). 2296 bytes result sent to driver 20/08/03 12:50:47.902 Executor task launch worker for task 275 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:47.902 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 86.0 (TID 274) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:47.902 Executor task launch worker for task 275 INFO Executor: Finished task 3.0 in stage 86.0 (TID 275). 2296 bytes result sent to driver 20/08/03 12:50:47.902 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 86.0 (TID 275) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:47.902 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 86.0, whose tasks have all completed, from pool 20/08/03 12:50:47.903 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 86 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:47.903 dag-scheduler-event-loop INFO DAGScheduler: Job 86 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.903 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 86: Stage finished 20/08/03 12:50:47.903 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 86 finished: treeAggregate at RDDLossFunction.scala:61, took 0.011010 s 20/08/03 12:50:47.903 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(168) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:47.904 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:47.904 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_168_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.904 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.706768 (rel: 4.93e-05) 0.0178151 20/08/03 12:50:47.904 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_170 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:47.905 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_170_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:47.906 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_170_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.906 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 170 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:47.912 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:47.913 dag-scheduler-event-loop INFO DAGScheduler: Got job 87 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:47.913 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 87 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:47.913 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.913 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.913 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 87 (MapPartitionsRDD[407] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:47.914 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_171 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.915 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_171_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:47.916 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_171_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:47.916 dag-scheduler-event-loop INFO SparkContext: Created broadcast 171 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.916 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 87 (MapPartitionsRDD[407] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:47.916 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 87.0 with 4 tasks 20/08/03 12:50:47.917 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 87.0 (TID 276, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.917 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 87.0 (TID 277, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.917 Executor task launch worker for task 276 INFO Executor: Running task 0.0 in stage 87.0 (TID 276) 20/08/03 12:50:47.917 Executor task launch worker for task 277 INFO Executor: Running task 1.0 in stage 87.0 (TID 277) 20/08/03 12:50:47.919 Executor task launch worker for task 276 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:47.919 Executor task launch worker for task 277 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:47.920 Executor task launch worker for task 276 INFO Executor: Finished task 0.0 in stage 87.0 (TID 276). 2339 bytes result sent to driver 20/08/03 12:50:47.920 Executor task launch worker for task 277 INFO Executor: Finished task 1.0 in stage 87.0 (TID 277). 2296 bytes result sent to driver 20/08/03 12:50:47.920 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 87.0 (TID 278, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.921 Executor task launch worker for task 278 INFO Executor: Running task 2.0 in stage 87.0 (TID 278) 20/08/03 12:50:47.921 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 87.0 (TID 279, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.921 Executor task launch worker for task 279 INFO Executor: Running task 3.0 in stage 87.0 (TID 279) 20/08/03 12:50:47.921 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 87.0 (TID 276) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:47.921 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 87.0 (TID 277) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:47.922 Executor task launch worker for task 278 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:47.922 Executor task launch worker for task 279 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:47.922 Executor task launch worker for task 278 INFO Executor: Finished task 2.0 in stage 87.0 (TID 278). 2296 bytes result sent to driver 20/08/03 12:50:47.922 Executor task launch worker for task 279 INFO Executor: Finished task 3.0 in stage 87.0 (TID 279). 2296 bytes result sent to driver 20/08/03 12:50:47.923 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 87.0 (TID 278) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:47.923 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 87.0 (TID 279) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:47.923 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 87.0, whose tasks have all completed, from pool 20/08/03 12:50:47.923 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 87 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:47.923 dag-scheduler-event-loop INFO DAGScheduler: Job 87 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.923 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 87: Stage finished 20/08/03 12:50:47.923 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 87 finished: treeAggregate at RDDLossFunction.scala:61, took 0.010980 s 20/08/03 12:50:47.924 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(170) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:47.924 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:47.924 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.706396 (rel: 0.000526) 0.0192487 20/08/03 12:50:47.924 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_170_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.925 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_172 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:47.926 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_172_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:47.926 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_172_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.926 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 172 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:47.933 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:47.933 dag-scheduler-event-loop INFO DAGScheduler: Got job 88 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:47.933 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 88 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:47.933 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.933 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.934 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 88 (MapPartitionsRDD[408] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:47.935 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_173 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.936 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_173_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:47.936 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_173_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:47.936 dag-scheduler-event-loop INFO SparkContext: Created broadcast 173 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.936 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 88 (MapPartitionsRDD[408] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:47.936 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 88.0 with 4 tasks 20/08/03 12:50:47.937 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 88.0 (TID 280, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.937 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 88.0 (TID 281, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.937 Executor task launch worker for task 280 INFO Executor: Running task 0.0 in stage 88.0 (TID 280) 20/08/03 12:50:47.937 Executor task launch worker for task 281 INFO Executor: Running task 1.0 in stage 88.0 (TID 281) 20/08/03 12:50:47.939 Executor task launch worker for task 281 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:47.939 Executor task launch worker for task 281 INFO Executor: Finished task 1.0 in stage 88.0 (TID 281). 2296 bytes result sent to driver 20/08/03 12:50:47.940 Executor task launch worker for task 280 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:47.940 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 88.0 (TID 282, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.940 Executor task launch worker for task 282 INFO Executor: Running task 2.0 in stage 88.0 (TID 282) 20/08/03 12:50:47.940 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 88.0 (TID 281) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:47.940 Executor task launch worker for task 280 INFO Executor: Finished task 0.0 in stage 88.0 (TID 280). 2296 bytes result sent to driver 20/08/03 12:50:47.940 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 88.0 (TID 283, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.940 Executor task launch worker for task 283 INFO Executor: Running task 3.0 in stage 88.0 (TID 283) 20/08/03 12:50:47.940 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 88.0 (TID 280) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:47.941 Executor task launch worker for task 282 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:47.942 Executor task launch worker for task 282 INFO Executor: Finished task 2.0 in stage 88.0 (TID 282). 2296 bytes result sent to driver 20/08/03 12:50:47.942 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 88.0 (TID 282) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:47.942 Executor task launch worker for task 283 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:47.943 Executor task launch worker for task 283 INFO Executor: Finished task 3.0 in stage 88.0 (TID 283). 2296 bytes result sent to driver 20/08/03 12:50:47.943 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 88.0 (TID 283) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:47.943 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 88.0, whose tasks have all completed, from pool 20/08/03 12:50:47.944 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 88 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:47.944 dag-scheduler-event-loop INFO DAGScheduler: Job 88 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.944 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 88: Stage finished 20/08/03 12:50:47.944 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 88 finished: treeAggregate at RDDLossFunction.scala:61, took 0.011295 s 20/08/03 12:50:47.944 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(172) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:47.945 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:47.945 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.704687 (rel: 0.00242) 0.0326162 20/08/03 12:50:47.945 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_172_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.945 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_174 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:47.947 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_174_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:47.947 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_174_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.947 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 174 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:47.953 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:47.954 dag-scheduler-event-loop INFO DAGScheduler: Got job 89 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:47.954 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 89 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:47.954 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.954 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.954 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 89 (MapPartitionsRDD[409] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:47.956 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_175 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.957 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_175_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:47.957 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_175_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:47.957 dag-scheduler-event-loop INFO SparkContext: Created broadcast 175 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.957 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 89 (MapPartitionsRDD[409] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:47.957 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 89.0 with 4 tasks 20/08/03 12:50:47.958 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 89.0 (TID 284, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.958 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 89.0 (TID 285, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.958 Executor task launch worker for task 284 INFO Executor: Running task 0.0 in stage 89.0 (TID 284) 20/08/03 12:50:47.958 Executor task launch worker for task 285 INFO Executor: Running task 1.0 in stage 89.0 (TID 285) 20/08/03 12:50:47.961 Executor task launch worker for task 284 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:47.961 Executor task launch worker for task 285 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:47.961 Executor task launch worker for task 284 INFO Executor: Finished task 0.0 in stage 89.0 (TID 284). 2339 bytes result sent to driver 20/08/03 12:50:47.961 Executor task launch worker for task 285 INFO Executor: Finished task 1.0 in stage 89.0 (TID 285). 2296 bytes result sent to driver 20/08/03 12:50:47.962 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 89.0 (TID 286, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.962 Executor task launch worker for task 286 INFO Executor: Running task 2.0 in stage 89.0 (TID 286) 20/08/03 12:50:47.962 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 89.0 (TID 287, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.962 Executor task launch worker for task 287 INFO Executor: Running task 3.0 in stage 89.0 (TID 287) 20/08/03 12:50:47.962 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 89.0 (TID 284) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:47.962 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 89.0 (TID 285) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:47.963 Executor task launch worker for task 286 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:47.964 Executor task launch worker for task 286 INFO Executor: Finished task 2.0 in stage 89.0 (TID 286). 2296 bytes result sent to driver 20/08/03 12:50:47.964 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 89.0 (TID 286) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:47.964 Executor task launch worker for task 287 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:47.965 Executor task launch worker for task 287 INFO Executor: Finished task 3.0 in stage 89.0 (TID 287). 2296 bytes result sent to driver 20/08/03 12:50:47.965 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 89.0 (TID 287) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:47.965 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 89.0, whose tasks have all completed, from pool 20/08/03 12:50:47.965 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 89 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:47.965 dag-scheduler-event-loop INFO DAGScheduler: Job 89 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.965 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 89: Stage finished 20/08/03 12:50:47.965 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 89 finished: treeAggregate at RDDLossFunction.scala:61, took 0.011926 s 20/08/03 12:50:47.966 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(174) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:47.966 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:47.966 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.703781 (rel: 0.00129) 0.0245661 20/08/03 12:50:47.966 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_174_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.967 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_176 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:47.968 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_176_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:47.968 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_176_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.968 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 176 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:47.975 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:47.975 dag-scheduler-event-loop INFO DAGScheduler: Got job 90 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:47.975 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 90 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:47.975 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:47.975 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:47.976 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 90 (MapPartitionsRDD[410] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:47.977 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_177 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:47.978 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_177_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:47.978 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_177_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:47.978 dag-scheduler-event-loop INFO SparkContext: Created broadcast 177 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:47.979 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 90 (MapPartitionsRDD[410] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:47.979 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 90.0 with 4 tasks 20/08/03 12:50:47.980 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 90.0 (TID 288, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.980 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 90.0 (TID 289, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.980 Executor task launch worker for task 289 INFO Executor: Running task 1.0 in stage 90.0 (TID 289) 20/08/03 12:50:47.980 Executor task launch worker for task 288 INFO Executor: Running task 0.0 in stage 90.0 (TID 288) 20/08/03 12:50:47.984 Executor task launch worker for task 288 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:47.984 Executor task launch worker for task 289 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:47.985 Executor task launch worker for task 289 INFO Executor: Finished task 1.0 in stage 90.0 (TID 289). 2339 bytes result sent to driver 20/08/03 12:50:47.985 Executor task launch worker for task 288 INFO Executor: Finished task 0.0 in stage 90.0 (TID 288). 2339 bytes result sent to driver 20/08/03 12:50:47.985 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 90.0 (TID 290, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.985 Executor task launch worker for task 290 INFO Executor: Running task 2.0 in stage 90.0 (TID 290) 20/08/03 12:50:47.985 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 90.0 (TID 291, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:47.986 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 90.0 (TID 289) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:47.986 Executor task launch worker for task 291 INFO Executor: Running task 3.0 in stage 90.0 (TID 291) 20/08/03 12:50:47.986 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 90.0 (TID 288) in 7 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:47.987 Executor task launch worker for task 290 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:47.987 Executor task launch worker for task 291 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:47.988 Executor task launch worker for task 291 INFO Executor: Finished task 3.0 in stage 90.0 (TID 291). 2296 bytes result sent to driver 20/08/03 12:50:47.988 Executor task launch worker for task 290 INFO Executor: Finished task 2.0 in stage 90.0 (TID 290). 2296 bytes result sent to driver 20/08/03 12:50:47.989 task-result-getter-2 INFO TaskSetManager: Finished task 3.0 in stage 90.0 (TID 291) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:47.989 task-result-getter-3 INFO TaskSetManager: Finished task 2.0 in stage 90.0 (TID 290) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:47.989 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 90.0, whose tasks have all completed, from pool 20/08/03 12:50:47.989 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 90 (treeAggregate at RDDLossFunction.scala:61) finished in 0.013 s 20/08/03 12:50:47.989 dag-scheduler-event-loop INFO DAGScheduler: Job 90 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:47.989 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 90: Stage finished 20/08/03 12:50:47.990 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 90 finished: treeAggregate at RDDLossFunction.scala:61, took 0.014732 s 20/08/03 12:50:47.990 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(176) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:47.991 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:47.991 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.703381 (rel: 0.000569) 0.00722326 20/08/03 12:50:47.991 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_176_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.992 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_178 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:47.993 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_178_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:47.993 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_178_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:47.994 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 178 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:48.001 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:48.001 dag-scheduler-event-loop INFO DAGScheduler: Got job 91 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:48.001 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 91 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:48.001 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:48.001 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:48.002 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 91 (MapPartitionsRDD[411] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:48.003 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_179 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:48.004 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_179_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:48.004 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_179_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.005 dag-scheduler-event-loop INFO SparkContext: Created broadcast 179 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:48.005 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 91 (MapPartitionsRDD[411] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:48.005 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 91.0 with 4 tasks 20/08/03 12:50:48.006 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 91.0 (TID 292, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.007 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 91.0 (TID 293, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.007 Executor task launch worker for task 292 INFO Executor: Running task 0.0 in stage 91.0 (TID 292) 20/08/03 12:50:48.007 Executor task launch worker for task 293 INFO Executor: Running task 1.0 in stage 91.0 (TID 293) 20/08/03 12:50:48.010 Executor task launch worker for task 292 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:48.010 Executor task launch worker for task 293 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:48.010 Executor task launch worker for task 293 INFO Executor: Finished task 1.0 in stage 91.0 (TID 293). 2296 bytes result sent to driver 20/08/03 12:50:48.010 Executor task launch worker for task 292 INFO Executor: Finished task 0.0 in stage 91.0 (TID 292). 2339 bytes result sent to driver 20/08/03 12:50:48.011 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 91.0 (TID 294, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.011 Executor task launch worker for task 294 INFO Executor: Running task 2.0 in stage 91.0 (TID 294) 20/08/03 12:50:48.011 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 91.0 (TID 295, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.011 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 91.0 (TID 293) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:48.011 Executor task launch worker for task 295 INFO Executor: Running task 3.0 in stage 91.0 (TID 295) 20/08/03 12:50:48.012 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 91.0 (TID 292) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:48.013 Executor task launch worker for task 294 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:48.013 Executor task launch worker for task 295 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:48.013 Executor task launch worker for task 294 INFO Executor: Finished task 2.0 in stage 91.0 (TID 294). 2296 bytes result sent to driver 20/08/03 12:50:48.013 Executor task launch worker for task 295 INFO Executor: Finished task 3.0 in stage 91.0 (TID 295). 2296 bytes result sent to driver 20/08/03 12:50:48.014 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 91.0 (TID 294) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:48.014 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 91.0 (TID 295) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:48.014 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 91.0, whose tasks have all completed, from pool 20/08/03 12:50:48.014 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 91 (treeAggregate at RDDLossFunction.scala:61) finished in 0.012 s 20/08/03 12:50:48.014 dag-scheduler-event-loop INFO DAGScheduler: Job 91 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:48.014 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 91: Stage finished 20/08/03 12:50:48.014 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 91 finished: treeAggregate at RDDLossFunction.scala:61, took 0.013769 s 20/08/03 12:50:48.015 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(178) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:48.015 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:48.016 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.703340 (rel: 5.79e-05) 0.00888468 20/08/03 12:50:48.016 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_178_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.016 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_180 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:48.017 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_180_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:48.018 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_180_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.018 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 180 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:48.026 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:48.026 dag-scheduler-event-loop INFO DAGScheduler: Got job 92 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:48.026 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 92 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:48.026 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:48.027 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:48.027 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 92 (MapPartitionsRDD[412] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:48.029 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_181 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:48.030 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_181_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:48.030 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_181_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.031 dag-scheduler-event-loop INFO SparkContext: Created broadcast 181 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:48.031 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 92 (MapPartitionsRDD[412] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:48.031 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 92.0 with 4 tasks 20/08/03 12:50:48.032 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 92.0 (TID 296, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.032 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 92.0 (TID 297, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.032 Executor task launch worker for task 297 INFO Executor: Running task 1.0 in stage 92.0 (TID 297) 20/08/03 12:50:48.032 Executor task launch worker for task 296 INFO Executor: Running task 0.0 in stage 92.0 (TID 296) 20/08/03 12:50:48.034 Executor task launch worker for task 297 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:48.035 Executor task launch worker for task 296 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:48.035 Executor task launch worker for task 297 INFO Executor: Finished task 1.0 in stage 92.0 (TID 297). 2296 bytes result sent to driver 20/08/03 12:50:48.035 Executor task launch worker for task 296 INFO Executor: Finished task 0.0 in stage 92.0 (TID 296). 2296 bytes result sent to driver 20/08/03 12:50:48.035 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 92.0 (TID 298, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.035 Executor task launch worker for task 298 INFO Executor: Running task 2.0 in stage 92.0 (TID 298) 20/08/03 12:50:48.036 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 92.0 (TID 299, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.036 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 92.0 (TID 297) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:48.036 Executor task launch worker for task 299 INFO Executor: Running task 3.0 in stage 92.0 (TID 299) 20/08/03 12:50:48.036 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 92.0 (TID 296) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:48.037 Executor task launch worker for task 299 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:48.038 Executor task launch worker for task 298 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:48.038 Executor task launch worker for task 299 INFO Executor: Finished task 3.0 in stage 92.0 (TID 299). 2296 bytes result sent to driver 20/08/03 12:50:48.038 Executor task launch worker for task 298 INFO Executor: Finished task 2.0 in stage 92.0 (TID 298). 2296 bytes result sent to driver 20/08/03 12:50:48.038 task-result-getter-2 INFO TaskSetManager: Finished task 3.0 in stage 92.0 (TID 299) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:48.039 task-result-getter-3 INFO TaskSetManager: Finished task 2.0 in stage 92.0 (TID 298) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:48.039 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 92.0, whose tasks have all completed, from pool 20/08/03 12:50:48.039 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 92 (treeAggregate at RDDLossFunction.scala:61) finished in 0.012 s 20/08/03 12:50:48.039 dag-scheduler-event-loop INFO DAGScheduler: Job 92 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:48.039 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 92: Stage finished 20/08/03 12:50:48.039 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 92 finished: treeAggregate at RDDLossFunction.scala:61, took 0.013598 s 20/08/03 12:50:48.040 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(180) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:48.040 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:48.041 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.703327 (rel: 1.88e-05) 0.00665647 20/08/03 12:50:48.041 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_180_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.041 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_182 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:48.043 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_182_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:48.043 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_182_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.043 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 182 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:48.050 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:48.050 dag-scheduler-event-loop INFO DAGScheduler: Got job 93 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:48.050 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 93 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:48.051 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:48.051 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:48.051 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 93 (MapPartitionsRDD[413] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:48.052 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_183 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:48.054 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_183_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:48.054 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_183_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.054 dag-scheduler-event-loop INFO SparkContext: Created broadcast 183 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:48.054 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 93 (MapPartitionsRDD[413] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:48.055 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 93.0 with 4 tasks 20/08/03 12:50:48.055 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 93.0 (TID 300, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.056 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 93.0 (TID 301, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.056 Executor task launch worker for task 300 INFO Executor: Running task 0.0 in stage 93.0 (TID 300) 20/08/03 12:50:48.056 Executor task launch worker for task 301 INFO Executor: Running task 1.0 in stage 93.0 (TID 301) 20/08/03 12:50:48.059 Executor task launch worker for task 300 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:48.059 Executor task launch worker for task 301 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:48.059 Executor task launch worker for task 300 INFO Executor: Finished task 0.0 in stage 93.0 (TID 300). 2339 bytes result sent to driver 20/08/03 12:50:48.059 Executor task launch worker for task 301 INFO Executor: Finished task 1.0 in stage 93.0 (TID 301). 2296 bytes result sent to driver 20/08/03 12:50:48.060 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 93.0 (TID 302, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.060 Executor task launch worker for task 302 INFO Executor: Running task 2.0 in stage 93.0 (TID 302) 20/08/03 12:50:48.060 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 93.0 (TID 303, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.060 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 93.0 (TID 300) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:48.060 Executor task launch worker for task 303 INFO Executor: Running task 3.0 in stage 93.0 (TID 303) 20/08/03 12:50:48.060 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 93.0 (TID 301) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:48.061 Executor task launch worker for task 302 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:48.062 Executor task launch worker for task 303 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:48.062 Executor task launch worker for task 302 INFO Executor: Finished task 2.0 in stage 93.0 (TID 302). 2296 bytes result sent to driver 20/08/03 12:50:48.062 Executor task launch worker for task 303 INFO Executor: Finished task 3.0 in stage 93.0 (TID 303). 2296 bytes result sent to driver 20/08/03 12:50:48.062 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 93.0 (TID 302) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:48.062 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 93.0 (TID 303) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:48.062 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 93.0, whose tasks have all completed, from pool 20/08/03 12:50:48.062 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 93 (treeAggregate at RDDLossFunction.scala:61) finished in 0.011 s 20/08/03 12:50:48.063 dag-scheduler-event-loop INFO DAGScheduler: Job 93 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:48.063 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 93: Stage finished 20/08/03 12:50:48.063 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 93 finished: treeAggregate at RDDLossFunction.scala:61, took 0.012662 s 20/08/03 12:50:48.063 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(182) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:48.064 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:48.064 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.703309 (rel: 2.51e-05) 0.00769572 20/08/03 12:50:48.064 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_182_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.064 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_184 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:48.065 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_184_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:48.066 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_184_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.066 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 184 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:48.073 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:48.073 dag-scheduler-event-loop INFO DAGScheduler: Got job 94 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:48.073 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 94 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:48.073 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:48.073 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:48.073 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 94 (MapPartitionsRDD[414] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:48.075 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_185 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:48.076 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_185_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:48.076 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_185_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.076 dag-scheduler-event-loop INFO SparkContext: Created broadcast 185 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:48.077 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 94 (MapPartitionsRDD[414] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:48.077 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 94.0 with 4 tasks 20/08/03 12:50:48.077 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 94.0 (TID 304, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.078 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 94.0 (TID 305, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.078 Executor task launch worker for task 304 INFO Executor: Running task 0.0 in stage 94.0 (TID 304) 20/08/03 12:50:48.078 Executor task launch worker for task 305 INFO Executor: Running task 1.0 in stage 94.0 (TID 305) 20/08/03 12:50:48.081 Executor task launch worker for task 305 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:48.081 Executor task launch worker for task 304 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:48.081 Executor task launch worker for task 304 INFO Executor: Finished task 0.0 in stage 94.0 (TID 304). 2296 bytes result sent to driver 20/08/03 12:50:48.081 Executor task launch worker for task 305 INFO Executor: Finished task 1.0 in stage 94.0 (TID 305). 2339 bytes result sent to driver 20/08/03 12:50:48.082 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 94.0 (TID 306, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.082 Executor task launch worker for task 306 INFO Executor: Running task 2.0 in stage 94.0 (TID 306) 20/08/03 12:50:48.082 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 94.0 (TID 307, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.082 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 94.0 (TID 304) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:48.082 Executor task launch worker for task 307 INFO Executor: Running task 3.0 in stage 94.0 (TID 307) 20/08/03 12:50:48.082 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 94.0 (TID 305) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:48.083 Executor task launch worker for task 306 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:48.084 Executor task launch worker for task 307 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:48.084 Executor task launch worker for task 306 INFO Executor: Finished task 2.0 in stage 94.0 (TID 306). 2296 bytes result sent to driver 20/08/03 12:50:48.084 Executor task launch worker for task 307 INFO Executor: Finished task 3.0 in stage 94.0 (TID 307). 2296 bytes result sent to driver 20/08/03 12:50:48.084 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 94.0 (TID 306) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:48.084 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 94.0 (TID 307) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:48.084 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 94.0, whose tasks have all completed, from pool 20/08/03 12:50:48.084 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 94 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:48.085 dag-scheduler-event-loop INFO DAGScheduler: Job 94 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:48.085 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 94: Stage finished 20/08/03 12:50:48.085 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 94 finished: treeAggregate at RDDLossFunction.scala:61, took 0.012017 s 20/08/03 12:50:48.085 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(184) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:48.085 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:48.086 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_184_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.086 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.703200 (rel: 0.000155) 0.0370874 20/08/03 12:50:48.086 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_186 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:48.087 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_186_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:48.087 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_186_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.088 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 186 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:48.094 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:48.095 dag-scheduler-event-loop INFO DAGScheduler: Got job 95 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:48.095 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 95 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:48.095 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:48.095 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:48.095 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 95 (MapPartitionsRDD[415] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:48.096 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_187 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:48.097 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_187_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:48.098 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_187_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.098 dag-scheduler-event-loop INFO SparkContext: Created broadcast 187 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:48.098 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 95 (MapPartitionsRDD[415] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:48.098 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 95.0 with 4 tasks 20/08/03 12:50:48.099 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 95.0 (TID 308, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.099 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 95.0 (TID 309, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.099 Executor task launch worker for task 309 INFO Executor: Running task 1.0 in stage 95.0 (TID 309) 20/08/03 12:50:48.099 Executor task launch worker for task 308 INFO Executor: Running task 0.0 in stage 95.0 (TID 308) 20/08/03 12:50:48.102 Executor task launch worker for task 308 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:48.102 Executor task launch worker for task 309 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:48.102 Executor task launch worker for task 309 INFO Executor: Finished task 1.0 in stage 95.0 (TID 309). 2296 bytes result sent to driver 20/08/03 12:50:48.102 Executor task launch worker for task 308 INFO Executor: Finished task 0.0 in stage 95.0 (TID 308). 2339 bytes result sent to driver 20/08/03 12:50:48.103 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 95.0 (TID 310, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.103 Executor task launch worker for task 310 INFO Executor: Running task 2.0 in stage 95.0 (TID 310) 20/08/03 12:50:48.103 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 95.0 (TID 311, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.103 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 95.0 (TID 309) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:48.103 Executor task launch worker for task 311 INFO Executor: Running task 3.0 in stage 95.0 (TID 311) 20/08/03 12:50:48.103 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 95.0 (TID 308) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:48.104 Executor task launch worker for task 310 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:48.105 Executor task launch worker for task 311 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:48.105 Executor task launch worker for task 310 INFO Executor: Finished task 2.0 in stage 95.0 (TID 310). 2296 bytes result sent to driver 20/08/03 12:50:48.105 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 95.0 (TID 310) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:48.105 Executor task launch worker for task 311 INFO Executor: Finished task 3.0 in stage 95.0 (TID 311). 2296 bytes result sent to driver 20/08/03 12:50:48.105 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 95.0 (TID 311) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:48.105 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 95.0, whose tasks have all completed, from pool 20/08/03 12:50:48.105 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 95 (treeAggregate at RDDLossFunction.scala:61) finished in 0.009 s 20/08/03 12:50:48.106 dag-scheduler-event-loop INFO DAGScheduler: Job 95 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:48.106 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 95: Stage finished 20/08/03 12:50:48.106 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 95 finished: treeAggregate at RDDLossFunction.scala:61, took 0.011340 s 20/08/03 12:50:48.106 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(186) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:48.107 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:48.107 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_186_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.107 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.703028 (rel: 0.000245) 0.0264371 20/08/03 12:50:48.107 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_188 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:48.108 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_188_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:48.109 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_188_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.109 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 188 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:48.115 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:48.116 dag-scheduler-event-loop INFO DAGScheduler: Got job 96 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:48.116 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 96 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:48.116 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:48.116 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:48.116 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 96 (MapPartitionsRDD[416] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:48.118 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_189 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:48.119 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_189_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:48.119 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_189_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.119 dag-scheduler-event-loop INFO SparkContext: Created broadcast 189 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:48.119 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 96 (MapPartitionsRDD[416] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:48.119 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 96.0 with 4 tasks 20/08/03 12:50:48.120 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 96.0 (TID 312, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.120 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 96.0 (TID 313, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.120 Executor task launch worker for task 312 INFO Executor: Running task 0.0 in stage 96.0 (TID 312) 20/08/03 12:50:48.120 Executor task launch worker for task 313 INFO Executor: Running task 1.0 in stage 96.0 (TID 313) 20/08/03 12:50:48.122 Executor task launch worker for task 312 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:48.122 Executor task launch worker for task 313 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:48.122 Executor task launch worker for task 312 INFO Executor: Finished task 0.0 in stage 96.0 (TID 312). 2296 bytes result sent to driver 20/08/03 12:50:48.122 Executor task launch worker for task 313 INFO Executor: Finished task 1.0 in stage 96.0 (TID 313). 2296 bytes result sent to driver 20/08/03 12:50:48.123 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 96.0 (TID 314, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.123 Executor task launch worker for task 314 INFO Executor: Running task 2.0 in stage 96.0 (TID 314) 20/08/03 12:50:48.123 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 96.0 (TID 315, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.123 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 96.0 (TID 312) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:48.123 Executor task launch worker for task 315 INFO Executor: Running task 3.0 in stage 96.0 (TID 315) 20/08/03 12:50:48.123 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 96.0 (TID 313) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:48.124 Executor task launch worker for task 314 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:48.125 Executor task launch worker for task 315 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:48.125 Executor task launch worker for task 314 INFO Executor: Finished task 2.0 in stage 96.0 (TID 314). 2296 bytes result sent to driver 20/08/03 12:50:48.125 Executor task launch worker for task 315 INFO Executor: Finished task 3.0 in stage 96.0 (TID 315). 2296 bytes result sent to driver 20/08/03 12:50:48.125 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 96.0 (TID 314) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:48.125 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 96.0 (TID 315) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:48.125 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 96.0, whose tasks have all completed, from pool 20/08/03 12:50:48.125 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 96 (treeAggregate at RDDLossFunction.scala:61) finished in 0.008 s 20/08/03 12:50:48.126 dag-scheduler-event-loop INFO DAGScheduler: Job 96 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:48.126 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 96: Stage finished 20/08/03 12:50:48.126 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 96 finished: treeAggregate at RDDLossFunction.scala:61, took 0.010249 s 20/08/03 12:50:48.126 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(188) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:48.126 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:48.127 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_188_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.127 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.702643 (rel: 0.000547) 0.0836470 20/08/03 12:50:48.127 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_190 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:48.128 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_190_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:48.129 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_190_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.129 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 190 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:48.135 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:48.136 dag-scheduler-event-loop INFO DAGScheduler: Got job 97 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:48.136 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 97 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:48.136 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:48.136 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:48.136 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 97 (MapPartitionsRDD[417] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:48.138 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_191 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:48.140 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_191_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:48.140 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_191_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.140 dag-scheduler-event-loop INFO SparkContext: Created broadcast 191 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:48.140 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 97 (MapPartitionsRDD[417] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:48.140 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 97.0 with 4 tasks 20/08/03 12:50:48.141 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 97.0 (TID 316, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.141 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 97.0 (TID 317, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.142 Executor task launch worker for task 316 INFO Executor: Running task 0.0 in stage 97.0 (TID 316) 20/08/03 12:50:48.142 Executor task launch worker for task 317 INFO Executor: Running task 1.0 in stage 97.0 (TID 317) 20/08/03 12:50:48.145 Executor task launch worker for task 317 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:48.145 Executor task launch worker for task 316 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:48.145 Executor task launch worker for task 316 INFO Executor: Finished task 0.0 in stage 97.0 (TID 316). 2296 bytes result sent to driver 20/08/03 12:50:48.145 Executor task launch worker for task 317 INFO Executor: Finished task 1.0 in stage 97.0 (TID 317). 2339 bytes result sent to driver 20/08/03 12:50:48.145 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 97.0 (TID 318, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.146 Executor task launch worker for task 318 INFO Executor: Running task 2.0 in stage 97.0 (TID 318) 20/08/03 12:50:48.146 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 97.0 (TID 319, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.146 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 97.0 (TID 316) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:48.146 Executor task launch worker for task 319 INFO Executor: Running task 3.0 in stage 97.0 (TID 319) 20/08/03 12:50:48.146 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 97.0 (TID 317) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:48.147 Executor task launch worker for task 318 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:48.147 Executor task launch worker for task 319 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:48.148 Executor task launch worker for task 318 INFO Executor: Finished task 2.0 in stage 97.0 (TID 318). 2296 bytes result sent to driver 20/08/03 12:50:48.148 Executor task launch worker for task 319 INFO Executor: Finished task 3.0 in stage 97.0 (TID 319). 2296 bytes result sent to driver 20/08/03 12:50:48.148 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 97.0 (TID 318) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:48.148 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 97.0 (TID 319) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:48.148 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 97.0, whose tasks have all completed, from pool 20/08/03 12:50:48.148 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 97 (treeAggregate at RDDLossFunction.scala:61) finished in 0.011 s 20/08/03 12:50:48.148 dag-scheduler-event-loop INFO DAGScheduler: Job 97 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:48.148 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 97: Stage finished 20/08/03 12:50:48.149 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 97 finished: treeAggregate at RDDLossFunction.scala:61, took 0.013080 s 20/08/03 12:50:48.149 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(190) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:48.149 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:48.150 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_190_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.150 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.702329 (rel: 0.000446) 0.0181781 20/08/03 12:50:48.150 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_192 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:48.151 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_192_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:48.152 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_192_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.152 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 192 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:48.159 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:48.160 dag-scheduler-event-loop INFO DAGScheduler: Got job 98 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:48.160 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 98 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:48.160 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:48.160 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:48.160 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 98 (MapPartitionsRDD[418] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:48.162 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_193 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:48.164 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_193_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:48.164 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_193_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.164 dag-scheduler-event-loop INFO SparkContext: Created broadcast 193 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:48.164 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 98 (MapPartitionsRDD[418] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:48.164 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 98.0 with 4 tasks 20/08/03 12:50:48.165 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 98.0 (TID 320, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.166 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 98.0 (TID 321, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.166 Executor task launch worker for task 321 INFO Executor: Running task 1.0 in stage 98.0 (TID 321) 20/08/03 12:50:48.166 Executor task launch worker for task 320 INFO Executor: Running task 0.0 in stage 98.0 (TID 320) 20/08/03 12:50:48.169 Executor task launch worker for task 320 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:48.169 Executor task launch worker for task 321 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:48.170 Executor task launch worker for task 320 INFO Executor: Finished task 0.0 in stage 98.0 (TID 320). 2296 bytes result sent to driver 20/08/03 12:50:48.170 Executor task launch worker for task 321 INFO Executor: Finished task 1.0 in stage 98.0 (TID 321). 2339 bytes result sent to driver 20/08/03 12:50:48.170 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 98.0 (TID 322, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.170 Executor task launch worker for task 322 INFO Executor: Running task 2.0 in stage 98.0 (TID 322) 20/08/03 12:50:48.187 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 98.0 (TID 323, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.188 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 98.0 (TID 320) in 22 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:48.188 Executor task launch worker for task 323 INFO Executor: Running task 3.0 in stage 98.0 (TID 323) 20/08/03 12:50:48.188 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 98.0 (TID 321) in 23 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:48.190 Executor task launch worker for task 322 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:48.190 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_135_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:48.190 Executor task launch worker for task 322 INFO Executor: Finished task 2.0 in stage 98.0 (TID 322). 2382 bytes result sent to driver 20/08/03 12:50:48.191 Executor task launch worker for task 323 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:48.191 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 98.0 (TID 322) in 21 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:48.191 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_131_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:48.191 Executor task launch worker for task 323 INFO Executor: Finished task 3.0 in stage 98.0 (TID 323). 2296 bytes result sent to driver 20/08/03 12:50:48.192 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 98.0 (TID 323) in 22 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:48.192 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 98.0, whose tasks have all completed, from pool 20/08/03 12:50:48.192 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 98 (treeAggregate at RDDLossFunction.scala:61) finished in 0.031 s 20/08/03 12:50:48.192 dag-scheduler-event-loop INFO DAGScheduler: Job 98 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:48.192 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 98: Stage finished 20/08/03 12:50:48.193 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 98 finished: treeAggregate at RDDLossFunction.scala:61, took 0.033062 s 20/08/03 12:50:48.193 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(192) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:48.193 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:48.194 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_141_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:48.194 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.702187 (rel: 0.000203) 0.0113743 20/08/03 12:50:48.194 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_192_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.194 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_175_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.194 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_194 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:48.196 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_144_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.4 KiB, free: 2.1 GiB) 20/08/03 12:50:48.196 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_194_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:48.196 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_194_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.197 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 194 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:48.197 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_123_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:48.198 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_185_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.199 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_159_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.200 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_137_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:48.201 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_157_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.201 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_125_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:48.202 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_179_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.203 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_126_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 6.4 KiB, free: 2.1 GiB) 20/08/03 12:50:48.205 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:48.205 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_177_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.205 dag-scheduler-event-loop INFO DAGScheduler: Got job 99 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:48.205 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 99 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:48.205 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:48.206 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:48.206 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 99 (MapPartitionsRDD[419] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:48.206 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_181_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.207 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_153_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.207 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_195 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:48.207 block-manager-slave-async-thread-pool-93 INFO BlockManager: Removing RDD 350 20/08/03 12:50:48.209 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_195_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:48.209 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_165_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.209 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_195_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.209 dag-scheduler-event-loop INFO SparkContext: Created broadcast 195 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:48.210 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 99 (MapPartitionsRDD[419] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:48.210 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 99.0 with 4 tasks 20/08/03 12:50:48.210 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_189_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.210 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 99.0 (TID 324, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.211 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 99.0 (TID 325, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.211 Executor task launch worker for task 324 INFO Executor: Running task 0.0 in stage 99.0 (TID 324) 20/08/03 12:50:48.211 Executor task launch worker for task 325 INFO Executor: Running task 1.0 in stage 99.0 (TID 325) 20/08/03 12:50:48.211 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_133_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:48.211 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_139_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:48.212 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_173_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.213 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_163_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.213 Executor task launch worker for task 324 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:48.213 Executor task launch worker for task 325 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:48.214 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_167_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.214 Executor task launch worker for task 324 INFO Executor: Finished task 0.0 in stage 99.0 (TID 324). 2296 bytes result sent to driver 20/08/03 12:50:48.214 Executor task launch worker for task 325 INFO Executor: Finished task 1.0 in stage 99.0 (TID 325). 2296 bytes result sent to driver 20/08/03 12:50:48.214 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 99.0 (TID 326, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.214 Executor task launch worker for task 326 INFO Executor: Running task 2.0 in stage 99.0 (TID 326) 20/08/03 12:50:48.214 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_145_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.4 KiB, free: 2.1 GiB) 20/08/03 12:50:48.214 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 99.0 (TID 327, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.214 Executor task launch worker for task 327 INFO Executor: Running task 3.0 in stage 99.0 (TID 327) 20/08/03 12:50:48.214 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 99.0 (TID 324) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:48.215 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 99.0 (TID 325) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:48.215 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_169_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.216 Executor task launch worker for task 326 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:48.216 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_120_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 6.4 KiB, free: 2.1 GiB) 20/08/03 12:50:48.216 Executor task launch worker for task 327 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:48.216 block-manager-slave-async-thread-pool-60 INFO BlockManager: Removing RDD 304 20/08/03 12:50:48.217 Executor task launch worker for task 326 INFO Executor: Finished task 2.0 in stage 99.0 (TID 326). 2296 bytes result sent to driver 20/08/03 12:50:48.217 Executor task launch worker for task 327 INFO Executor: Finished task 3.0 in stage 99.0 (TID 327). 2296 bytes result sent to driver 20/08/03 12:50:48.217 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 99.0 (TID 326) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:48.217 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 99.0 (TID 327) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:48.217 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 99.0, whose tasks have all completed, from pool 20/08/03 12:50:48.217 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_183_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.217 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 99 (treeAggregate at RDDLossFunction.scala:61) finished in 0.011 s 20/08/03 12:50:48.217 dag-scheduler-event-loop INFO DAGScheduler: Job 99 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:48.217 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 99: Stage finished 20/08/03 12:50:48.218 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 99 finished: treeAggregate at RDDLossFunction.scala:61, took 0.012718 s 20/08/03 12:50:48.218 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_191_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.218 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(194) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:48.218 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_151_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.218 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:48.218 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_194_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.218 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.702168 (rel: 2.61e-05) 0.00149523 20/08/03 12:50:48.219 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_171_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.219 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_146_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.8 KiB, free: 2.1 GiB) 20/08/03 12:50:48.219 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_196 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:48.220 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_143_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:48.220 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_196_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:48.220 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_196_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.221 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 196 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:48.221 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_187_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.221 block-manager-slave-async-thread-pool-5 INFO BlockManager: Removing RDD 283 20/08/03 12:50:48.222 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_149_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.222 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_155_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.223 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_129_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 5.9 KiB, free: 2.1 GiB) 20/08/03 12:50:48.224 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_161_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.227 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:48.228 dag-scheduler-event-loop INFO DAGScheduler: Got job 100 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:48.228 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 100 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:48.228 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:48.228 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:48.228 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 100 (MapPartitionsRDD[420] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:48.229 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_197 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:48.230 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_197_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:48.231 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_197_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.231 dag-scheduler-event-loop INFO SparkContext: Created broadcast 197 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:48.231 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 100 (MapPartitionsRDD[420] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:48.231 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 100.0 with 4 tasks 20/08/03 12:50:48.232 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 100.0 (TID 328, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.232 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 100.0 (TID 329, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.232 Executor task launch worker for task 328 INFO Executor: Running task 0.0 in stage 100.0 (TID 328) 20/08/03 12:50:48.232 Executor task launch worker for task 329 INFO Executor: Running task 1.0 in stage 100.0 (TID 329) 20/08/03 12:50:48.234 Executor task launch worker for task 328 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:48.234 Executor task launch worker for task 329 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:48.235 Executor task launch worker for task 328 INFO Executor: Finished task 0.0 in stage 100.0 (TID 328). 2296 bytes result sent to driver 20/08/03 12:50:48.235 Executor task launch worker for task 329 INFO Executor: Finished task 1.0 in stage 100.0 (TID 329). 2296 bytes result sent to driver 20/08/03 12:50:48.235 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 100.0 (TID 330, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.235 Executor task launch worker for task 330 INFO Executor: Running task 2.0 in stage 100.0 (TID 330) 20/08/03 12:50:48.235 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 100.0 (TID 331, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.235 Executor task launch worker for task 331 INFO Executor: Running task 3.0 in stage 100.0 (TID 331) 20/08/03 12:50:48.235 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 100.0 (TID 328) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:48.235 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 100.0 (TID 329) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:48.236 Executor task launch worker for task 330 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:48.237 Executor task launch worker for task 330 INFO Executor: Finished task 2.0 in stage 100.0 (TID 330). 2296 bytes result sent to driver 20/08/03 12:50:48.237 Executor task launch worker for task 331 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:48.237 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 100.0 (TID 330) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:48.237 Executor task launch worker for task 331 INFO Executor: Finished task 3.0 in stage 100.0 (TID 331). 2296 bytes result sent to driver 20/08/03 12:50:48.237 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 100.0 (TID 331) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:48.238 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 100.0, whose tasks have all completed, from pool 20/08/03 12:50:48.238 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 100 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:48.238 dag-scheduler-event-loop INFO DAGScheduler: Job 100 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:48.238 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 100: Stage finished 20/08/03 12:50:48.238 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 100 finished: treeAggregate at RDDLossFunction.scala:61, took 0.010723 s 20/08/03 12:50:48.238 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(196) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:48.239 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:48.239 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.702168 (rel: 5.52e-07) 0.000903472 20/08/03 12:50:48.239 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_196_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.240 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_198 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:48.241 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_198_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:48.241 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_198_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.241 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 198 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:48.248 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:48.248 dag-scheduler-event-loop INFO DAGScheduler: Got job 101 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:48.248 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 101 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:48.248 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:48.249 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:48.249 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 101 (MapPartitionsRDD[421] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:48.250 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_199 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:48.251 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_199_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:48.251 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_199_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.252 dag-scheduler-event-loop INFO SparkContext: Created broadcast 199 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:48.252 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 101 (MapPartitionsRDD[421] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:48.252 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 101.0 with 4 tasks 20/08/03 12:50:48.252 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 101.0 (TID 332, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.253 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 101.0 (TID 333, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.253 Executor task launch worker for task 332 INFO Executor: Running task 0.0 in stage 101.0 (TID 332) 20/08/03 12:50:48.253 Executor task launch worker for task 333 INFO Executor: Running task 1.0 in stage 101.0 (TID 333) 20/08/03 12:50:48.255 Executor task launch worker for task 333 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:48.255 Executor task launch worker for task 332 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:48.255 Executor task launch worker for task 333 INFO Executor: Finished task 1.0 in stage 101.0 (TID 333). 2296 bytes result sent to driver 20/08/03 12:50:48.255 Executor task launch worker for task 332 INFO Executor: Finished task 0.0 in stage 101.0 (TID 332). 2296 bytes result sent to driver 20/08/03 12:50:48.256 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 101.0 (TID 334, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.256 Executor task launch worker for task 334 INFO Executor: Running task 2.0 in stage 101.0 (TID 334) 20/08/03 12:50:48.256 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 101.0 (TID 335, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.256 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 101.0 (TID 333) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:48.256 Executor task launch worker for task 335 INFO Executor: Running task 3.0 in stage 101.0 (TID 335) 20/08/03 12:50:48.256 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 101.0 (TID 332) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:48.258 Executor task launch worker for task 334 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:48.258 Executor task launch worker for task 335 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:48.258 Executor task launch worker for task 334 INFO Executor: Finished task 2.0 in stage 101.0 (TID 334). 2296 bytes result sent to driver 20/08/03 12:50:48.258 Executor task launch worker for task 335 INFO Executor: Finished task 3.0 in stage 101.0 (TID 335). 2296 bytes result sent to driver 20/08/03 12:50:48.258 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 101.0 (TID 334) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:48.259 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 101.0 (TID 335) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:48.259 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 101.0, whose tasks have all completed, from pool 20/08/03 12:50:48.259 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 101 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:48.259 dag-scheduler-event-loop INFO DAGScheduler: Job 101 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:48.259 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 101: Stage finished 20/08/03 12:50:48.259 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 101 finished: treeAggregate at RDDLossFunction.scala:61, took 0.011219 s 20/08/03 12:50:48.260 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(198) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:48.260 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:48.260 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.702167 (rel: 8.45e-07) 0.00230020 20/08/03 12:50:48.260 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_198_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.261 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_200 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:48.262 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_200_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:48.262 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_200_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.262 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 200 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:48.268 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:48.269 dag-scheduler-event-loop INFO DAGScheduler: Got job 102 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:48.269 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 102 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:48.269 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:48.269 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:48.269 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 102 (MapPartitionsRDD[422] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:48.270 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_201 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:48.271 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_201_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:48.272 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_201_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.272 dag-scheduler-event-loop INFO SparkContext: Created broadcast 201 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:48.272 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 102 (MapPartitionsRDD[422] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:48.272 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 102.0 with 4 tasks 20/08/03 12:50:48.273 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 102.0 (TID 336, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.273 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 102.0 (TID 337, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.273 Executor task launch worker for task 337 INFO Executor: Running task 1.0 in stage 102.0 (TID 337) 20/08/03 12:50:48.273 Executor task launch worker for task 336 INFO Executor: Running task 0.0 in stage 102.0 (TID 336) 20/08/03 12:50:48.275 Executor task launch worker for task 337 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:48.275 Executor task launch worker for task 336 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:48.276 Executor task launch worker for task 337 INFO Executor: Finished task 1.0 in stage 102.0 (TID 337). 2296 bytes result sent to driver 20/08/03 12:50:48.276 Executor task launch worker for task 336 INFO Executor: Finished task 0.0 in stage 102.0 (TID 336). 2296 bytes result sent to driver 20/08/03 12:50:48.276 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 102.0 (TID 338, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.276 Executor task launch worker for task 338 INFO Executor: Running task 2.0 in stage 102.0 (TID 338) 20/08/03 12:50:48.277 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 102.0 (TID 339, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:48.277 Executor task launch worker for task 339 INFO Executor: Running task 3.0 in stage 102.0 (TID 339) 20/08/03 12:50:48.277 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 102.0 (TID 337) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:48.277 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 102.0 (TID 336) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:48.278 Executor task launch worker for task 338 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:48.278 Executor task launch worker for task 339 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:48.278 Executor task launch worker for task 338 INFO Executor: Finished task 2.0 in stage 102.0 (TID 338). 2296 bytes result sent to driver 20/08/03 12:50:48.278 Executor task launch worker for task 339 INFO Executor: Finished task 3.0 in stage 102.0 (TID 339). 2296 bytes result sent to driver 20/08/03 12:50:48.279 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 102.0 (TID 338) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:48.279 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 102.0 (TID 339) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:48.279 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 102.0, whose tasks have all completed, from pool 20/08/03 12:50:48.279 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 102 (treeAggregate at RDDLossFunction.scala:61) finished in 0.009 s 20/08/03 12:50:48.279 dag-scheduler-event-loop INFO DAGScheduler: Job 102 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:48.279 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 102: Stage finished 20/08/03 12:50:48.279 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 102 finished: treeAggregate at RDDLossFunction.scala:61, took 0.010868 s 20/08/03 12:50:48.280 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(200) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:48.280 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:48.280 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.702166 (rel: 1.30e-06) 0.00388042 20/08/03 12:50:48.280 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_200_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.281 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_202 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:48.282 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_202_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:48.282 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_202_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:48.282 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 202 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:48.288 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:48.289 dag-scheduler-event-loop INFO DAGScheduler: Got job 103 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:48.289 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 103 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:48.289 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:48.289 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:48.289 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 103 (MapPartitionsRDD[423] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:48.290 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_203 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:48.291 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_203_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:48.292 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_203_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:48.292 dag-scheduler-event-loop INFO SparkContext: Created broadcast 203 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:48.292 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 103 (MapPartitionsRDD[423] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:48.292 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 103.0 with 4 tasks 20/08/03 12:50:48.293 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 103.0 (TID 340, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.058 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 103.0 (TID 341, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.059 Executor task launch worker for task 341 INFO Executor: Running task 1.0 in stage 103.0 (TID 341) 20/08/03 12:50:49.059 Executor task launch worker for task 340 INFO Executor: Running task 0.0 in stage 103.0 (TID 340) 20/08/03 12:50:49.063 Executor task launch worker for task 341 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.064 Executor task launch worker for task 341 INFO Executor: Finished task 1.0 in stage 103.0 (TID 341). 2296 bytes result sent to driver 20/08/03 12:50:49.065 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 103.0 (TID 342, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.065 Executor task launch worker for task 340 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.065 Executor task launch worker for task 342 INFO Executor: Running task 2.0 in stage 103.0 (TID 342) 20/08/03 12:50:49.065 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 103.0 (TID 341) in 7 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.066 Executor task launch worker for task 340 INFO Executor: Finished task 0.0 in stage 103.0 (TID 340). 2339 bytes result sent to driver 20/08/03 12:50:49.066 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 103.0 (TID 343, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.067 Executor task launch worker for task 343 INFO Executor: Running task 3.0 in stage 103.0 (TID 343) 20/08/03 12:50:49.067 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 103.0 (TID 340) in 775 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.069 Executor task launch worker for task 342 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.069 Executor task launch worker for task 343 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.069 Executor task launch worker for task 343 INFO Executor: Finished task 3.0 in stage 103.0 (TID 343). 2296 bytes result sent to driver 20/08/03 12:50:49.070 Executor task launch worker for task 342 INFO Executor: Finished task 2.0 in stage 103.0 (TID 342). 2296 bytes result sent to driver 20/08/03 12:50:49.070 task-result-getter-2 INFO TaskSetManager: Finished task 3.0 in stage 103.0 (TID 343) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.071 task-result-getter-3 INFO TaskSetManager: Finished task 2.0 in stage 103.0 (TID 342) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.071 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 103.0, whose tasks have all completed, from pool 20/08/03 12:50:49.071 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 103 (treeAggregate at RDDLossFunction.scala:61) finished in 0.781 s 20/08/03 12:50:49.071 dag-scheduler-event-loop INFO DAGScheduler: Job 103 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.071 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 103: Stage finished 20/08/03 12:50:49.071 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 103 finished: treeAggregate at RDDLossFunction.scala:61, took 0.783055 s 20/08/03 12:50:49.072 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(202) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.072 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.073 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.702164 (rel: 3.32e-06) 0.00577897 20/08/03 12:50:49.073 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_202_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.073 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_204 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.075 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_204_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.075 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_204_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.075 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 204 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.082 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.082 dag-scheduler-event-loop INFO DAGScheduler: Got job 104 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.082 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 104 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.082 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.083 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.083 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 104 (MapPartitionsRDD[424] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.084 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_205 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.085 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_205_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.085 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_205_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.086 dag-scheduler-event-loop INFO SparkContext: Created broadcast 205 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.086 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 104 (MapPartitionsRDD[424] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.086 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 104.0 with 4 tasks 20/08/03 12:50:49.087 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 104.0 (TID 344, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.087 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 104.0 (TID 345, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.087 Executor task launch worker for task 344 INFO Executor: Running task 0.0 in stage 104.0 (TID 344) 20/08/03 12:50:49.087 Executor task launch worker for task 345 INFO Executor: Running task 1.0 in stage 104.0 (TID 345) 20/08/03 12:50:49.089 Executor task launch worker for task 345 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.089 Executor task launch worker for task 344 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.090 Executor task launch worker for task 345 INFO Executor: Finished task 1.0 in stage 104.0 (TID 345). 2296 bytes result sent to driver 20/08/03 12:50:49.090 Executor task launch worker for task 344 INFO Executor: Finished task 0.0 in stage 104.0 (TID 344). 2296 bytes result sent to driver 20/08/03 12:50:49.090 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 104.0 (TID 346, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.090 Executor task launch worker for task 346 INFO Executor: Running task 2.0 in stage 104.0 (TID 346) 20/08/03 12:50:49.091 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 104.0 (TID 347, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.091 Executor task launch worker for task 347 INFO Executor: Running task 3.0 in stage 104.0 (TID 347) 20/08/03 12:50:49.091 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 104.0 (TID 345) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.091 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 104.0 (TID 344) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.092 Executor task launch worker for task 346 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.092 Executor task launch worker for task 347 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.092 Executor task launch worker for task 346 INFO Executor: Finished task 2.0 in stage 104.0 (TID 346). 2296 bytes result sent to driver 20/08/03 12:50:49.092 Executor task launch worker for task 347 INFO Executor: Finished task 3.0 in stage 104.0 (TID 347). 2296 bytes result sent to driver 20/08/03 12:50:49.093 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 104.0 (TID 346) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.093 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 104.0 (TID 347) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.093 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 104.0, whose tasks have all completed, from pool 20/08/03 12:50:49.093 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 104 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:49.093 dag-scheduler-event-loop INFO DAGScheduler: Job 104 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.093 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 104: Stage finished 20/08/03 12:50:49.093 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 104 finished: treeAggregate at RDDLossFunction.scala:61, took 0.011485 s 20/08/03 12:50:49.094 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(204) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.094 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.094 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.702160 (rel: 5.86e-06) 0.00642929 20/08/03 12:50:49.094 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_204_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.095 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_206 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.096 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_206_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.096 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_206_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.096 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 206 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.103 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.103 dag-scheduler-event-loop INFO DAGScheduler: Got job 105 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.103 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 105 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.103 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.104 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.104 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 105 (MapPartitionsRDD[425] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.105 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_207 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.256 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_207_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.257 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_207_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.257 dag-scheduler-event-loop INFO SparkContext: Created broadcast 207 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.258 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 105 (MapPartitionsRDD[425] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.258 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 105.0 with 4 tasks 20/08/03 12:50:49.259 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 105.0 (TID 348, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.260 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 105.0 (TID 349, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.260 Executor task launch worker for task 349 INFO Executor: Running task 1.0 in stage 105.0 (TID 349) 20/08/03 12:50:49.260 Executor task launch worker for task 348 INFO Executor: Running task 0.0 in stage 105.0 (TID 348) 20/08/03 12:50:49.263 Executor task launch worker for task 349 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.263 Executor task launch worker for task 349 INFO Executor: Finished task 1.0 in stage 105.0 (TID 349). 2296 bytes result sent to driver 20/08/03 12:50:49.264 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 105.0 (TID 350, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.264 Executor task launch worker for task 350 INFO Executor: Running task 2.0 in stage 105.0 (TID 350) 20/08/03 12:50:49.264 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 105.0 (TID 349) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.264 Executor task launch worker for task 348 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.265 Executor task launch worker for task 348 INFO Executor: Finished task 0.0 in stage 105.0 (TID 348). 2339 bytes result sent to driver 20/08/03 12:50:49.265 Executor task launch worker for task 350 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.265 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 105.0 (TID 351, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.265 Executor task launch worker for task 351 INFO Executor: Running task 3.0 in stage 105.0 (TID 351) 20/08/03 12:50:49.265 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 105.0 (TID 348) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.266 Executor task launch worker for task 350 INFO Executor: Finished task 2.0 in stage 105.0 (TID 350). 2296 bytes result sent to driver 20/08/03 12:50:49.266 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 105.0 (TID 350) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.267 Executor task launch worker for task 351 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.267 Executor task launch worker for task 351 INFO Executor: Finished task 3.0 in stage 105.0 (TID 351). 2296 bytes result sent to driver 20/08/03 12:50:49.268 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 105.0 (TID 351) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.268 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 105.0, whose tasks have all completed, from pool 20/08/03 12:50:49.268 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 105 (treeAggregate at RDDLossFunction.scala:61) finished in 0.164 s 20/08/03 12:50:49.268 dag-scheduler-event-loop INFO DAGScheduler: Job 105 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.268 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 105: Stage finished 20/08/03 12:50:49.268 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 105 finished: treeAggregate at RDDLossFunction.scala:61, took 0.165375 s 20/08/03 12:50:49.269 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(206) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.270 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_206_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.270 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_208 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.271 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_208_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.271 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_208_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.271 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 208 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.277 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.278 dag-scheduler-event-loop INFO DAGScheduler: Got job 106 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.278 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 106 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.278 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.278 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.279 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 106 (MapPartitionsRDD[426] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.280 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_209 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.282 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_209_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.282 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_209_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.282 dag-scheduler-event-loop INFO SparkContext: Created broadcast 209 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.283 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 106 (MapPartitionsRDD[426] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.283 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 106.0 with 4 tasks 20/08/03 12:50:49.284 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 106.0 (TID 352, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.284 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 106.0 (TID 353, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.284 Executor task launch worker for task 353 INFO Executor: Running task 1.0 in stage 106.0 (TID 353) 20/08/03 12:50:49.284 Executor task launch worker for task 352 INFO Executor: Running task 0.0 in stage 106.0 (TID 352) 20/08/03 12:50:49.286 Executor task launch worker for task 353 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.286 Executor task launch worker for task 352 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.287 Executor task launch worker for task 353 INFO Executor: Finished task 1.0 in stage 106.0 (TID 353). 2296 bytes result sent to driver 20/08/03 12:50:49.287 Executor task launch worker for task 352 INFO Executor: Finished task 0.0 in stage 106.0 (TID 352). 2296 bytes result sent to driver 20/08/03 12:50:49.287 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 106.0 (TID 354, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.287 Executor task launch worker for task 354 INFO Executor: Running task 2.0 in stage 106.0 (TID 354) 20/08/03 12:50:49.287 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 106.0 (TID 355, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.287 Executor task launch worker for task 355 INFO Executor: Running task 3.0 in stage 106.0 (TID 355) 20/08/03 12:50:49.287 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 106.0 (TID 353) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.288 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 106.0 (TID 352) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.289 Executor task launch worker for task 354 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.289 Executor task launch worker for task 355 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.289 Executor task launch worker for task 354 INFO Executor: Finished task 2.0 in stage 106.0 (TID 354). 2296 bytes result sent to driver 20/08/03 12:50:49.289 Executor task launch worker for task 355 INFO Executor: Finished task 3.0 in stage 106.0 (TID 355). 2296 bytes result sent to driver 20/08/03 12:50:49.289 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 106.0 (TID 354) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.290 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 106.0 (TID 355) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.290 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 106.0, whose tasks have all completed, from pool 20/08/03 12:50:49.290 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 106 (treeAggregate at RDDLossFunction.scala:61) finished in 0.011 s 20/08/03 12:50:49.290 dag-scheduler-event-loop INFO DAGScheduler: Job 106 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.290 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 106: Stage finished 20/08/03 12:50:49.290 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 106 finished: treeAggregate at RDDLossFunction.scala:61, took 0.012556 s 20/08/03 12:50:49.291 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(208) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.291 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO StrongWolfeLineSearch: Line search t: 0.16930476062421762 fval: 0.7021574107499159 rhs: 0.7021599158017666 cdd: 5.670797078850777E-10 20/08/03 12:50:49.291 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 0.1693 20/08/03 12:50:49.291 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.702157 (rel: 3.57e-06) 0.0146308 20/08/03 12:50:49.291 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_208_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.292 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_210 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.293 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_210_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.293 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_210_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.294 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 210 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.300 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.301 dag-scheduler-event-loop INFO DAGScheduler: Got job 107 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.301 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 107 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.301 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.301 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.301 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 107 (MapPartitionsRDD[427] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.303 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_211 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.304 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_211_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.304 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_211_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.304 dag-scheduler-event-loop INFO SparkContext: Created broadcast 211 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.304 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 107 (MapPartitionsRDD[427] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.304 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 107.0 with 4 tasks 20/08/03 12:50:49.305 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 107.0 (TID 356, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.305 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 107.0 (TID 357, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.305 Executor task launch worker for task 357 INFO Executor: Running task 1.0 in stage 107.0 (TID 357) 20/08/03 12:50:49.305 Executor task launch worker for task 356 INFO Executor: Running task 0.0 in stage 107.0 (TID 356) 20/08/03 12:50:49.307 Executor task launch worker for task 356 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.307 Executor task launch worker for task 357 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.308 Executor task launch worker for task 356 INFO Executor: Finished task 0.0 in stage 107.0 (TID 356). 2296 bytes result sent to driver 20/08/03 12:50:49.308 Executor task launch worker for task 357 INFO Executor: Finished task 1.0 in stage 107.0 (TID 357). 2296 bytes result sent to driver 20/08/03 12:50:49.308 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 107.0 (TID 358, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.308 Executor task launch worker for task 358 INFO Executor: Running task 2.0 in stage 107.0 (TID 358) 20/08/03 12:50:49.308 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 107.0 (TID 359, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.308 Executor task launch worker for task 359 INFO Executor: Running task 3.0 in stage 107.0 (TID 359) 20/08/03 12:50:49.308 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 107.0 (TID 356) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.309 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 107.0 (TID 357) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.310 Executor task launch worker for task 358 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.310 Executor task launch worker for task 358 INFO Executor: Finished task 2.0 in stage 107.0 (TID 358). 2296 bytes result sent to driver 20/08/03 12:50:49.310 Executor task launch worker for task 359 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.310 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 107.0 (TID 358) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.310 Executor task launch worker for task 359 INFO Executor: Finished task 3.0 in stage 107.0 (TID 359). 2296 bytes result sent to driver 20/08/03 12:50:49.311 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 107.0 (TID 359) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.311 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 107.0, whose tasks have all completed, from pool 20/08/03 12:50:49.311 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 107 (treeAggregate at RDDLossFunction.scala:61) finished in 0.009 s 20/08/03 12:50:49.311 dag-scheduler-event-loop INFO DAGScheduler: Job 107 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.311 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 107: Stage finished 20/08/03 12:50:49.311 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 107 finished: treeAggregate at RDDLossFunction.scala:61, took 0.010839 s 20/08/03 12:50:49.312 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(210) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.312 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.312 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.702147 (rel: 1.42e-05) 0.0116257 20/08/03 12:50:49.312 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_210_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.313 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_212 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.314 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_212_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.314 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_212_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.314 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 212 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.321 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.321 dag-scheduler-event-loop INFO DAGScheduler: Got job 108 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.321 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 108 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.321 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.321 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.322 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 108 (MapPartitionsRDD[428] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.323 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_213 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.324 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_213_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.324 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_213_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.325 dag-scheduler-event-loop INFO SparkContext: Created broadcast 213 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.325 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 108 (MapPartitionsRDD[428] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.325 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 108.0 with 4 tasks 20/08/03 12:50:49.326 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 108.0 (TID 360, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.326 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 108.0 (TID 361, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.326 Executor task launch worker for task 361 INFO Executor: Running task 1.0 in stage 108.0 (TID 361) 20/08/03 12:50:49.326 Executor task launch worker for task 360 INFO Executor: Running task 0.0 in stage 108.0 (TID 360) 20/08/03 12:50:49.328 Executor task launch worker for task 360 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.328 Executor task launch worker for task 361 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.329 Executor task launch worker for task 360 INFO Executor: Finished task 0.0 in stage 108.0 (TID 360). 2296 bytes result sent to driver 20/08/03 12:50:49.329 Executor task launch worker for task 361 INFO Executor: Finished task 1.0 in stage 108.0 (TID 361). 2296 bytes result sent to driver 20/08/03 12:50:49.329 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 108.0 (TID 362, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.329 Executor task launch worker for task 362 INFO Executor: Running task 2.0 in stage 108.0 (TID 362) 20/08/03 12:50:49.329 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 108.0 (TID 363, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.330 Executor task launch worker for task 363 INFO Executor: Running task 3.0 in stage 108.0 (TID 363) 20/08/03 12:50:49.330 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 108.0 (TID 360) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.330 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 108.0 (TID 361) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.331 Executor task launch worker for task 362 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.331 Executor task launch worker for task 363 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.331 Executor task launch worker for task 362 INFO Executor: Finished task 2.0 in stage 108.0 (TID 362). 2296 bytes result sent to driver 20/08/03 12:50:49.331 Executor task launch worker for task 363 INFO Executor: Finished task 3.0 in stage 108.0 (TID 363). 2296 bytes result sent to driver 20/08/03 12:50:49.332 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 108.0 (TID 362) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.332 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 108.0 (TID 363) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.332 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 108.0, whose tasks have all completed, from pool 20/08/03 12:50:49.332 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 108 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:49.332 dag-scheduler-event-loop INFO DAGScheduler: Job 108 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.332 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 108: Stage finished 20/08/03 12:50:49.332 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 108 finished: treeAggregate at RDDLossFunction.scala:61, took 0.011715 s 20/08/03 12:50:49.333 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(212) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.333 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.333 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.702119 (rel: 4.06e-05) 0.00692901 20/08/03 12:50:49.333 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_212_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.334 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_214 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.335 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_214_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.335 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_214_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.335 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 214 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.342 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.342 dag-scheduler-event-loop INFO DAGScheduler: Got job 109 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.342 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 109 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.342 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.343 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.343 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 109 (MapPartitionsRDD[429] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.344 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_215 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.345 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_215_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.345 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_215_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.345 dag-scheduler-event-loop INFO SparkContext: Created broadcast 215 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.346 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 109 (MapPartitionsRDD[429] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.346 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 109.0 with 4 tasks 20/08/03 12:50:49.347 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 109.0 (TID 364, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.347 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 109.0 (TID 365, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.347 Executor task launch worker for task 364 INFO Executor: Running task 0.0 in stage 109.0 (TID 364) 20/08/03 12:50:49.347 Executor task launch worker for task 365 INFO Executor: Running task 1.0 in stage 109.0 (TID 365) 20/08/03 12:50:49.349 Executor task launch worker for task 364 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.349 Executor task launch worker for task 365 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.349 Executor task launch worker for task 364 INFO Executor: Finished task 0.0 in stage 109.0 (TID 364). 2296 bytes result sent to driver 20/08/03 12:50:49.349 Executor task launch worker for task 365 INFO Executor: Finished task 1.0 in stage 109.0 (TID 365). 2296 bytes result sent to driver 20/08/03 12:50:49.350 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 109.0 (TID 366, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.350 Executor task launch worker for task 366 INFO Executor: Running task 2.0 in stage 109.0 (TID 366) 20/08/03 12:50:49.350 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 109.0 (TID 367, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.350 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 109.0 (TID 364) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.350 Executor task launch worker for task 367 INFO Executor: Running task 3.0 in stage 109.0 (TID 367) 20/08/03 12:50:49.350 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 109.0 (TID 365) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.351 Executor task launch worker for task 366 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.351 Executor task launch worker for task 367 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.352 Executor task launch worker for task 366 INFO Executor: Finished task 2.0 in stage 109.0 (TID 366). 2296 bytes result sent to driver 20/08/03 12:50:49.352 Executor task launch worker for task 367 INFO Executor: Finished task 3.0 in stage 109.0 (TID 367). 2296 bytes result sent to driver 20/08/03 12:50:49.352 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 109.0 (TID 366) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.352 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 109.0 (TID 367) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.352 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 109.0, whose tasks have all completed, from pool 20/08/03 12:50:49.352 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 109 (treeAggregate at RDDLossFunction.scala:61) finished in 0.009 s 20/08/03 12:50:49.353 dag-scheduler-event-loop INFO DAGScheduler: Job 109 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.353 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 109: Stage finished 20/08/03 12:50:49.353 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 109 finished: treeAggregate at RDDLossFunction.scala:61, took 0.010849 s 20/08/03 12:50:49.353 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(214) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.354 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.354 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.702073 (rel: 6.60e-05) 0.0122200 20/08/03 12:50:49.354 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_214_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.354 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_216 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.355 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_216_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.355 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_216_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.356 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 216 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.363 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.363 dag-scheduler-event-loop INFO DAGScheduler: Got job 110 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.363 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 110 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.363 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.363 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.364 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 110 (MapPartitionsRDD[430] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.365 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_217 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.366 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_217_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.366 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_217_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.366 dag-scheduler-event-loop INFO SparkContext: Created broadcast 217 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.367 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 110 (MapPartitionsRDD[430] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.367 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 110.0 with 4 tasks 20/08/03 12:50:49.368 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 110.0 (TID 368, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.368 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 110.0 (TID 369, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.368 Executor task launch worker for task 368 INFO Executor: Running task 0.0 in stage 110.0 (TID 368) 20/08/03 12:50:49.368 Executor task launch worker for task 369 INFO Executor: Running task 1.0 in stage 110.0 (TID 369) 20/08/03 12:50:49.370 Executor task launch worker for task 369 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.370 Executor task launch worker for task 368 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.371 Executor task launch worker for task 369 INFO Executor: Finished task 1.0 in stage 110.0 (TID 369). 2296 bytes result sent to driver 20/08/03 12:50:49.371 Executor task launch worker for task 368 INFO Executor: Finished task 0.0 in stage 110.0 (TID 368). 2296 bytes result sent to driver 20/08/03 12:50:49.371 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 110.0 (TID 370, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.371 Executor task launch worker for task 370 INFO Executor: Running task 2.0 in stage 110.0 (TID 370) 20/08/03 12:50:49.371 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 110.0 (TID 371, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.372 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 110.0 (TID 369) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.372 Executor task launch worker for task 371 INFO Executor: Running task 3.0 in stage 110.0 (TID 371) 20/08/03 12:50:49.372 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 110.0 (TID 368) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.373 Executor task launch worker for task 370 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.373 Executor task launch worker for task 370 INFO Executor: Finished task 2.0 in stage 110.0 (TID 370). 2296 bytes result sent to driver 20/08/03 12:50:49.373 Executor task launch worker for task 371 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.373 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 110.0 (TID 370) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.373 Executor task launch worker for task 371 INFO Executor: Finished task 3.0 in stage 110.0 (TID 371). 2296 bytes result sent to driver 20/08/03 12:50:49.374 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 110.0 (TID 371) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.374 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 110.0, whose tasks have all completed, from pool 20/08/03 12:50:49.374 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 110 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:49.374 dag-scheduler-event-loop INFO DAGScheduler: Job 110 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.374 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 110: Stage finished 20/08/03 12:50:49.374 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 110 finished: treeAggregate at RDDLossFunction.scala:61, took 0.011608 s 20/08/03 12:50:49.375 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(216) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.375 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.375 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.702004 (rel: 9.74e-05) 0.0212014 20/08/03 12:50:49.375 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_216_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.376 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_218 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.377 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_218_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.377 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_218_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.377 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 218 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.384 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.384 dag-scheduler-event-loop INFO DAGScheduler: Got job 111 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.384 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 111 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.384 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.385 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.385 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 111 (MapPartitionsRDD[431] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.386 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_219 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.387 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_219_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.387 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_219_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.387 dag-scheduler-event-loop INFO SparkContext: Created broadcast 219 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.388 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 111 (MapPartitionsRDD[431] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.388 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 111.0 with 4 tasks 20/08/03 12:50:49.388 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 111.0 (TID 372, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.389 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 111.0 (TID 373, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.389 Executor task launch worker for task 373 INFO Executor: Running task 1.0 in stage 111.0 (TID 373) 20/08/03 12:50:49.389 Executor task launch worker for task 372 INFO Executor: Running task 0.0 in stage 111.0 (TID 372) 20/08/03 12:50:49.391 Executor task launch worker for task 373 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.391 Executor task launch worker for task 372 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.391 Executor task launch worker for task 373 INFO Executor: Finished task 1.0 in stage 111.0 (TID 373). 2296 bytes result sent to driver 20/08/03 12:50:49.392 Executor task launch worker for task 372 INFO Executor: Finished task 0.0 in stage 111.0 (TID 372). 2296 bytes result sent to driver 20/08/03 12:50:49.392 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 111.0 (TID 374, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.392 Executor task launch worker for task 374 INFO Executor: Running task 2.0 in stage 111.0 (TID 374) 20/08/03 12:50:49.392 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 111.0 (TID 375, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.392 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 111.0 (TID 373) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.392 Executor task launch worker for task 375 INFO Executor: Running task 3.0 in stage 111.0 (TID 375) 20/08/03 12:50:49.392 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 111.0 (TID 372) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.394 Executor task launch worker for task 374 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.394 Executor task launch worker for task 374 INFO Executor: Finished task 2.0 in stage 111.0 (TID 374). 2296 bytes result sent to driver 20/08/03 12:50:49.394 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 111.0 (TID 374) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.395 Executor task launch worker for task 375 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.395 Executor task launch worker for task 375 INFO Executor: Finished task 3.0 in stage 111.0 (TID 375). 2296 bytes result sent to driver 20/08/03 12:50:49.396 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 111.0 (TID 375) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.396 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 111.0, whose tasks have all completed, from pool 20/08/03 12:50:49.396 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 111 (treeAggregate at RDDLossFunction.scala:61) finished in 0.011 s 20/08/03 12:50:49.396 dag-scheduler-event-loop INFO DAGScheduler: Job 111 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.396 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 111: Stage finished 20/08/03 12:50:49.396 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 111 finished: treeAggregate at RDDLossFunction.scala:61, took 0.012510 s 20/08/03 12:50:49.397 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(218) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.397 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.398 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.701911 (rel: 0.000133) 0.0547783 20/08/03 12:50:49.398 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_218_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.398 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_220 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.399 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_220_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.400 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_220_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.400 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 220 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.407 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.407 dag-scheduler-event-loop INFO DAGScheduler: Got job 112 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.407 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 112 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.407 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.408 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.408 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 112 (MapPartitionsRDD[432] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.409 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_221 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.410 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_221_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.411 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_221_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.411 dag-scheduler-event-loop INFO SparkContext: Created broadcast 221 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.411 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 112 (MapPartitionsRDD[432] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.411 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 112.0 with 4 tasks 20/08/03 12:50:49.412 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 112.0 (TID 376, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.412 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 112.0 (TID 377, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.412 Executor task launch worker for task 376 INFO Executor: Running task 0.0 in stage 112.0 (TID 376) 20/08/03 12:50:49.412 Executor task launch worker for task 377 INFO Executor: Running task 1.0 in stage 112.0 (TID 377) 20/08/03 12:50:49.415 Executor task launch worker for task 377 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.415 Executor task launch worker for task 377 INFO Executor: Finished task 1.0 in stage 112.0 (TID 377). 2296 bytes result sent to driver 20/08/03 12:50:49.416 Executor task launch worker for task 376 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.416 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 112.0 (TID 378, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.416 Executor task launch worker for task 378 INFO Executor: Running task 2.0 in stage 112.0 (TID 378) 20/08/03 12:50:49.416 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 112.0 (TID 377) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.416 Executor task launch worker for task 376 INFO Executor: Finished task 0.0 in stage 112.0 (TID 376). 2296 bytes result sent to driver 20/08/03 12:50:49.417 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 112.0 (TID 379, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.417 Executor task launch worker for task 379 INFO Executor: Running task 3.0 in stage 112.0 (TID 379) 20/08/03 12:50:49.417 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 112.0 (TID 376) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.418 Executor task launch worker for task 378 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.418 Executor task launch worker for task 378 INFO Executor: Finished task 2.0 in stage 112.0 (TID 378). 2296 bytes result sent to driver 20/08/03 12:50:49.418 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 112.0 (TID 378) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.419 Executor task launch worker for task 379 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.419 Executor task launch worker for task 379 INFO Executor: Finished task 3.0 in stage 112.0 (TID 379). 2296 bytes result sent to driver 20/08/03 12:50:49.420 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 112.0 (TID 379) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.420 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 112.0, whose tasks have all completed, from pool 20/08/03 12:50:49.420 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 112 (treeAggregate at RDDLossFunction.scala:61) finished in 0.012 s 20/08/03 12:50:49.420 dag-scheduler-event-loop INFO DAGScheduler: Job 112 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.420 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 112: Stage finished 20/08/03 12:50:49.420 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 112 finished: treeAggregate at RDDLossFunction.scala:61, took 0.013368 s 20/08/03 12:50:49.421 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(220) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.421 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.421 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.701769 (rel: 0.000202) 0.0386704 20/08/03 12:50:49.421 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_220_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.422 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_222 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.423 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_222_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.423 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_222_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.423 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 222 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.430 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.431 dag-scheduler-event-loop INFO DAGScheduler: Got job 113 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.431 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 113 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.431 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.431 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.431 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 113 (MapPartitionsRDD[433] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.432 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_223 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.433 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_223_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.433 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_223_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.434 dag-scheduler-event-loop INFO SparkContext: Created broadcast 223 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.434 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 113 (MapPartitionsRDD[433] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.434 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 113.0 with 4 tasks 20/08/03 12:50:49.435 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 113.0 (TID 380, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.435 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 113.0 (TID 381, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.436 Executor task launch worker for task 380 INFO Executor: Running task 0.0 in stage 113.0 (TID 380) 20/08/03 12:50:49.436 Executor task launch worker for task 381 INFO Executor: Running task 1.0 in stage 113.0 (TID 381) 20/08/03 12:50:49.438 Executor task launch worker for task 380 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.438 Executor task launch worker for task 381 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.438 Executor task launch worker for task 380 INFO Executor: Finished task 0.0 in stage 113.0 (TID 380). 2296 bytes result sent to driver 20/08/03 12:50:49.438 Executor task launch worker for task 381 INFO Executor: Finished task 1.0 in stage 113.0 (TID 381). 2296 bytes result sent to driver 20/08/03 12:50:49.439 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 113.0 (TID 382, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.439 Executor task launch worker for task 382 INFO Executor: Running task 2.0 in stage 113.0 (TID 382) 20/08/03 12:50:49.439 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 113.0 (TID 383, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.439 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 113.0 (TID 380) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.439 Executor task launch worker for task 383 INFO Executor: Running task 3.0 in stage 113.0 (TID 383) 20/08/03 12:50:49.439 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 113.0 (TID 381) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.441 Executor task launch worker for task 382 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.441 Executor task launch worker for task 382 INFO Executor: Finished task 2.0 in stage 113.0 (TID 382). 2296 bytes result sent to driver 20/08/03 12:50:49.441 Executor task launch worker for task 383 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.441 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 113.0 (TID 382) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.442 Executor task launch worker for task 383 INFO Executor: Finished task 3.0 in stage 113.0 (TID 383). 2296 bytes result sent to driver 20/08/03 12:50:49.442 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 113.0 (TID 383) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.442 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 113.0, whose tasks have all completed, from pool 20/08/03 12:50:49.442 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 113 (treeAggregate at RDDLossFunction.scala:61) finished in 0.011 s 20/08/03 12:50:49.442 dag-scheduler-event-loop INFO DAGScheduler: Job 113 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.442 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 113: Stage finished 20/08/03 12:50:49.442 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 113 finished: treeAggregate at RDDLossFunction.scala:61, took 0.012338 s 20/08/03 12:50:49.443 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(222) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.443 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.443 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.701630 (rel: 0.000197) 0.0305899 20/08/03 12:50:49.444 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_222_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.444 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_224 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.445 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_224_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.445 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_224_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.445 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 224 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.452 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.453 dag-scheduler-event-loop INFO DAGScheduler: Got job 114 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.453 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 114 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.453 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.453 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.453 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 114 (MapPartitionsRDD[434] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.455 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_225 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.456 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_225_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.456 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_225_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.456 dag-scheduler-event-loop INFO SparkContext: Created broadcast 225 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.457 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 114 (MapPartitionsRDD[434] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.457 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 114.0 with 4 tasks 20/08/03 12:50:49.458 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 114.0 (TID 384, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.458 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 114.0 (TID 385, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.458 Executor task launch worker for task 384 INFO Executor: Running task 0.0 in stage 114.0 (TID 384) 20/08/03 12:50:49.458 Executor task launch worker for task 385 INFO Executor: Running task 1.0 in stage 114.0 (TID 385) 20/08/03 12:50:49.461 Executor task launch worker for task 384 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.461 Executor task launch worker for task 385 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.461 Executor task launch worker for task 384 INFO Executor: Finished task 0.0 in stage 114.0 (TID 384). 2296 bytes result sent to driver 20/08/03 12:50:49.462 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 114.0 (TID 386, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.462 Executor task launch worker for task 385 INFO Executor: Finished task 1.0 in stage 114.0 (TID 385). 2296 bytes result sent to driver 20/08/03 12:50:49.462 Executor task launch worker for task 386 INFO Executor: Running task 2.0 in stage 114.0 (TID 386) 20/08/03 12:50:49.462 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 114.0 (TID 387, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.462 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 114.0 (TID 384) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.462 Executor task launch worker for task 387 INFO Executor: Running task 3.0 in stage 114.0 (TID 387) 20/08/03 12:50:49.462 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 114.0 (TID 385) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.464 Executor task launch worker for task 387 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.464 Executor task launch worker for task 386 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.464 Executor task launch worker for task 387 INFO Executor: Finished task 3.0 in stage 114.0 (TID 387). 2296 bytes result sent to driver 20/08/03 12:50:49.465 Executor task launch worker for task 386 INFO Executor: Finished task 2.0 in stage 114.0 (TID 386). 2296 bytes result sent to driver 20/08/03 12:50:49.465 task-result-getter-2 INFO TaskSetManager: Finished task 3.0 in stage 114.0 (TID 387) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.465 task-result-getter-3 INFO TaskSetManager: Finished task 2.0 in stage 114.0 (TID 386) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.465 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 114.0, whose tasks have all completed, from pool 20/08/03 12:50:49.465 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 114 (treeAggregate at RDDLossFunction.scala:61) finished in 0.011 s 20/08/03 12:50:49.466 dag-scheduler-event-loop INFO DAGScheduler: Job 114 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.466 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 114: Stage finished 20/08/03 12:50:49.466 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 114 finished: treeAggregate at RDDLossFunction.scala:61, took 0.013510 s 20/08/03 12:50:49.466 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(224) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.467 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_224_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.467 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_226 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.468 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_226_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.468 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_226_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.468 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 226 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.475 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.476 dag-scheduler-event-loop INFO DAGScheduler: Got job 115 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.476 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 115 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.476 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.476 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.476 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 115 (MapPartitionsRDD[435] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.478 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_227 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.479 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_227_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.479 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_227_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.479 dag-scheduler-event-loop INFO SparkContext: Created broadcast 227 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.480 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 115 (MapPartitionsRDD[435] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.480 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 115.0 with 4 tasks 20/08/03 12:50:49.480 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 115.0 (TID 388, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.480 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 115.0 (TID 389, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.481 Executor task launch worker for task 389 INFO Executor: Running task 1.0 in stage 115.0 (TID 389) 20/08/03 12:50:49.481 Executor task launch worker for task 388 INFO Executor: Running task 0.0 in stage 115.0 (TID 388) 20/08/03 12:50:49.483 Executor task launch worker for task 389 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.483 Executor task launch worker for task 388 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.483 Executor task launch worker for task 389 INFO Executor: Finished task 1.0 in stage 115.0 (TID 389). 2296 bytes result sent to driver 20/08/03 12:50:49.483 Executor task launch worker for task 388 INFO Executor: Finished task 0.0 in stage 115.0 (TID 388). 2296 bytes result sent to driver 20/08/03 12:50:49.484 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 115.0 (TID 390, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.484 Executor task launch worker for task 390 INFO Executor: Running task 2.0 in stage 115.0 (TID 390) 20/08/03 12:50:49.484 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 115.0 (TID 391, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.484 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 115.0 (TID 389) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.484 Executor task launch worker for task 391 INFO Executor: Running task 3.0 in stage 115.0 (TID 391) 20/08/03 12:50:49.484 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 115.0 (TID 388) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.486 Executor task launch worker for task 390 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.486 Executor task launch worker for task 391 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.486 Executor task launch worker for task 390 INFO Executor: Finished task 2.0 in stage 115.0 (TID 390). 2296 bytes result sent to driver 20/08/03 12:50:49.486 Executor task launch worker for task 391 INFO Executor: Finished task 3.0 in stage 115.0 (TID 391). 2296 bytes result sent to driver 20/08/03 12:50:49.486 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 115.0 (TID 390) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.486 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 115.0 (TID 391) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.486 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 115.0, whose tasks have all completed, from pool 20/08/03 12:50:49.487 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 115 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:49.487 dag-scheduler-event-loop INFO DAGScheduler: Job 115 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.487 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 115: Stage finished 20/08/03 12:50:49.487 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 115 finished: treeAggregate at RDDLossFunction.scala:61, took 0.011550 s 20/08/03 12:50:49.487 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(226) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.488 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO StrongWolfeLineSearch: Line search t: 0.29563200757887187 fval: 0.7015536967552849 rhs: 0.7016304320196503 cdd: 7.84660996078386E-8 20/08/03 12:50:49.488 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 0.2956 20/08/03 12:50:49.488 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.701554 (rel: 0.000109) 0.0138103 20/08/03 12:50:49.488 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_226_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.488 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_228 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.489 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_228_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.490 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_228_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.490 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 228 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.496 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.497 dag-scheduler-event-loop INFO DAGScheduler: Got job 116 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.497 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 116 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.497 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.497 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.497 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 116 (MapPartitionsRDD[436] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.498 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_229 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.499 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_229_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.499 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_229_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.500 dag-scheduler-event-loop INFO SparkContext: Created broadcast 229 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.500 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 116 (MapPartitionsRDD[436] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.500 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 116.0 with 4 tasks 20/08/03 12:50:49.500 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 116.0 (TID 392, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.501 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 116.0 (TID 393, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.501 Executor task launch worker for task 392 INFO Executor: Running task 0.0 in stage 116.0 (TID 392) 20/08/03 12:50:49.501 Executor task launch worker for task 393 INFO Executor: Running task 1.0 in stage 116.0 (TID 393) 20/08/03 12:50:49.503 Executor task launch worker for task 392 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.503 Executor task launch worker for task 393 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.503 Executor task launch worker for task 392 INFO Executor: Finished task 0.0 in stage 116.0 (TID 392). 2296 bytes result sent to driver 20/08/03 12:50:49.503 Executor task launch worker for task 393 INFO Executor: Finished task 1.0 in stage 116.0 (TID 393). 2296 bytes result sent to driver 20/08/03 12:50:49.504 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 116.0 (TID 394, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.504 Executor task launch worker for task 394 INFO Executor: Running task 2.0 in stage 116.0 (TID 394) 20/08/03 12:50:49.504 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 116.0 (TID 395, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.504 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 116.0 (TID 392) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.504 Executor task launch worker for task 395 INFO Executor: Running task 3.0 in stage 116.0 (TID 395) 20/08/03 12:50:49.504 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 116.0 (TID 393) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.505 Executor task launch worker for task 394 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.506 Executor task launch worker for task 394 INFO Executor: Finished task 2.0 in stage 116.0 (TID 394). 2296 bytes result sent to driver 20/08/03 12:50:49.506 Executor task launch worker for task 395 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.506 Executor task launch worker for task 395 INFO Executor: Finished task 3.0 in stage 116.0 (TID 395). 2296 bytes result sent to driver 20/08/03 12:50:49.507 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 116.0 (TID 394) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.507 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 116.0 (TID 395) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.507 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 116.0, whose tasks have all completed, from pool 20/08/03 12:50:49.507 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 116 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:49.507 dag-scheduler-event-loop INFO DAGScheduler: Job 116 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.507 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 116: Stage finished 20/08/03 12:50:49.507 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 116 finished: treeAggregate at RDDLossFunction.scala:61, took 0.011178 s 20/08/03 12:50:49.508 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(228) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.508 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.508 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.701483 (rel: 0.000101) 0.00773242 20/08/03 12:50:49.508 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_228_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.509 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_230 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.510 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_230_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.510 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_230_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.510 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 230 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.517 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.517 dag-scheduler-event-loop INFO DAGScheduler: Got job 117 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.517 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 117 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.517 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.517 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.517 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 117 (MapPartitionsRDD[437] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.519 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_231 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.520 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_231_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.520 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_231_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.520 dag-scheduler-event-loop INFO SparkContext: Created broadcast 231 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.521 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 117 (MapPartitionsRDD[437] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.521 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 117.0 with 4 tasks 20/08/03 12:50:49.522 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 117.0 (TID 396, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.522 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 117.0 (TID 397, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.522 Executor task launch worker for task 396 INFO Executor: Running task 0.0 in stage 117.0 (TID 396) 20/08/03 12:50:49.522 Executor task launch worker for task 397 INFO Executor: Running task 1.0 in stage 117.0 (TID 397) 20/08/03 12:50:49.524 Executor task launch worker for task 396 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.524 Executor task launch worker for task 397 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.524 Executor task launch worker for task 396 INFO Executor: Finished task 0.0 in stage 117.0 (TID 396). 2296 bytes result sent to driver 20/08/03 12:50:49.525 Executor task launch worker for task 397 INFO Executor: Finished task 1.0 in stage 117.0 (TID 397). 2296 bytes result sent to driver 20/08/03 12:50:49.525 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 117.0 (TID 398, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.525 Executor task launch worker for task 398 INFO Executor: Running task 2.0 in stage 117.0 (TID 398) 20/08/03 12:50:49.525 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 117.0 (TID 399, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.525 Executor task launch worker for task 399 INFO Executor: Running task 3.0 in stage 117.0 (TID 399) 20/08/03 12:50:49.525 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 117.0 (TID 396) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.525 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 117.0 (TID 397) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.526 Executor task launch worker for task 398 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.527 Executor task launch worker for task 399 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.527 Executor task launch worker for task 398 INFO Executor: Finished task 2.0 in stage 117.0 (TID 398). 2296 bytes result sent to driver 20/08/03 12:50:49.527 Executor task launch worker for task 399 INFO Executor: Finished task 3.0 in stage 117.0 (TID 399). 2296 bytes result sent to driver 20/08/03 12:50:49.527 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 117.0 (TID 398) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.527 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 117.0 (TID 399) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.527 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 117.0, whose tasks have all completed, from pool 20/08/03 12:50:49.527 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 117 (treeAggregate at RDDLossFunction.scala:61) finished in 0.009 s 20/08/03 12:50:49.528 dag-scheduler-event-loop INFO DAGScheduler: Job 117 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.528 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 117: Stage finished 20/08/03 12:50:49.528 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 117 finished: treeAggregate at RDDLossFunction.scala:61, took 0.011144 s 20/08/03 12:50:49.528 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(230) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.529 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.529 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.701455 (rel: 3.93e-05) 0.0117929 20/08/03 12:50:49.529 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_230_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.529 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_232 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.530 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_232_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.530 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_232_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.531 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 232 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.537 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.538 dag-scheduler-event-loop INFO DAGScheduler: Got job 118 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.538 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 118 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.538 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.538 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.538 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 118 (MapPartitionsRDD[438] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.539 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_233 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.540 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_233_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.541 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_233_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.541 dag-scheduler-event-loop INFO SparkContext: Created broadcast 233 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.541 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 118 (MapPartitionsRDD[438] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.541 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 118.0 with 4 tasks 20/08/03 12:50:49.542 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 118.0 (TID 400, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.542 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 118.0 (TID 401, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.542 Executor task launch worker for task 400 INFO Executor: Running task 0.0 in stage 118.0 (TID 400) 20/08/03 12:50:49.542 Executor task launch worker for task 401 INFO Executor: Running task 1.0 in stage 118.0 (TID 401) 20/08/03 12:50:49.544 Executor task launch worker for task 401 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.544 Executor task launch worker for task 400 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.544 Executor task launch worker for task 401 INFO Executor: Finished task 1.0 in stage 118.0 (TID 401). 2296 bytes result sent to driver 20/08/03 12:50:49.545 Executor task launch worker for task 400 INFO Executor: Finished task 0.0 in stage 118.0 (TID 400). 2296 bytes result sent to driver 20/08/03 12:50:49.545 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 118.0 (TID 402, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.545 Executor task launch worker for task 402 INFO Executor: Running task 2.0 in stage 118.0 (TID 402) 20/08/03 12:50:49.545 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 118.0 (TID 403, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.545 Executor task launch worker for task 403 INFO Executor: Running task 3.0 in stage 118.0 (TID 403) 20/08/03 12:50:49.545 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 118.0 (TID 401) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.545 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 118.0 (TID 400) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.546 Executor task launch worker for task 402 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.547 Executor task launch worker for task 403 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.547 Executor task launch worker for task 402 INFO Executor: Finished task 2.0 in stage 118.0 (TID 402). 2296 bytes result sent to driver 20/08/03 12:50:49.547 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 118.0 (TID 402) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.547 Executor task launch worker for task 403 INFO Executor: Finished task 3.0 in stage 118.0 (TID 403). 2296 bytes result sent to driver 20/08/03 12:50:49.547 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 118.0 (TID 403) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.547 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 118.0, whose tasks have all completed, from pool 20/08/03 12:50:49.548 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 118 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:49.548 dag-scheduler-event-loop INFO DAGScheduler: Job 118 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.548 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 118: Stage finished 20/08/03 12:50:49.548 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 118 finished: treeAggregate at RDDLossFunction.scala:61, took 0.010880 s 20/08/03 12:50:49.549 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(232) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.549 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_232_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.549 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_234 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.550 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_234_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.551 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_234_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.551 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 234 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.557 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.558 dag-scheduler-event-loop INFO DAGScheduler: Got job 119 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.558 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 119 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.558 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.558 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.558 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 119 (MapPartitionsRDD[439] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.560 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_235 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.561 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_235_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.561 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_235_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.561 dag-scheduler-event-loop INFO SparkContext: Created broadcast 235 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.562 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 119 (MapPartitionsRDD[439] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.562 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 119.0 with 4 tasks 20/08/03 12:50:49.562 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 119.0 (TID 404, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.563 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 119.0 (TID 405, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.563 Executor task launch worker for task 405 INFO Executor: Running task 1.0 in stage 119.0 (TID 405) 20/08/03 12:50:49.563 Executor task launch worker for task 404 INFO Executor: Running task 0.0 in stage 119.0 (TID 404) 20/08/03 12:50:49.564 Executor task launch worker for task 404 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.565 Executor task launch worker for task 405 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.565 Executor task launch worker for task 404 INFO Executor: Finished task 0.0 in stage 119.0 (TID 404). 2296 bytes result sent to driver 20/08/03 12:50:49.565 Executor task launch worker for task 405 INFO Executor: Finished task 1.0 in stage 119.0 (TID 405). 2296 bytes result sent to driver 20/08/03 12:50:49.565 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 119.0 (TID 406, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.565 Executor task launch worker for task 406 INFO Executor: Running task 2.0 in stage 119.0 (TID 406) 20/08/03 12:50:49.565 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 119.0 (TID 407, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.566 Executor task launch worker for task 407 INFO Executor: Running task 3.0 in stage 119.0 (TID 407) 20/08/03 12:50:49.566 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 119.0 (TID 404) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.566 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 119.0 (TID 405) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.567 Executor task launch worker for task 406 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.567 Executor task launch worker for task 406 INFO Executor: Finished task 2.0 in stage 119.0 (TID 406). 2296 bytes result sent to driver 20/08/03 12:50:49.567 Executor task launch worker for task 407 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.567 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 119.0 (TID 406) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.567 Executor task launch worker for task 407 INFO Executor: Finished task 3.0 in stage 119.0 (TID 407). 2296 bytes result sent to driver 20/08/03 12:50:49.568 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 119.0 (TID 407) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.568 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 119.0, whose tasks have all completed, from pool 20/08/03 12:50:49.568 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 119 (treeAggregate at RDDLossFunction.scala:61) finished in 0.009 s 20/08/03 12:50:49.568 dag-scheduler-event-loop INFO DAGScheduler: Job 119 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.568 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 119: Stage finished 20/08/03 12:50:49.568 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 119 finished: treeAggregate at RDDLossFunction.scala:61, took 0.010672 s 20/08/03 12:50:49.569 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(234) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.569 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO StrongWolfeLineSearch: Line search t: 0.1 fval: 0.7014539526876473 rhs: 0.7014551786388885 cdd: 1.615510930624041E-7 20/08/03 12:50:49.569 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 0.1000 20/08/03 12:50:49.569 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_234_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.569 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.701454 (rel: 1.75e-06) 0.0151704 20/08/03 12:50:49.570 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_236 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.571 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_236_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.571 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_236_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.571 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 236 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.578 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.578 dag-scheduler-event-loop INFO DAGScheduler: Got job 120 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.578 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 120 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.578 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.578 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.579 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 120 (MapPartitionsRDD[440] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.580 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_237 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.581 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_237_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.581 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_237_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.582 dag-scheduler-event-loop INFO SparkContext: Created broadcast 237 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.582 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 120 (MapPartitionsRDD[440] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.582 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 120.0 with 4 tasks 20/08/03 12:50:49.583 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 120.0 (TID 408, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.583 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 120.0 (TID 409, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.583 Executor task launch worker for task 409 INFO Executor: Running task 1.0 in stage 120.0 (TID 409) 20/08/03 12:50:49.583 Executor task launch worker for task 408 INFO Executor: Running task 0.0 in stage 120.0 (TID 408) 20/08/03 12:50:49.586 Executor task launch worker for task 409 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.586 Executor task launch worker for task 408 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.586 Executor task launch worker for task 409 INFO Executor: Finished task 1.0 in stage 120.0 (TID 409). 2296 bytes result sent to driver 20/08/03 12:50:49.587 Executor task launch worker for task 408 INFO Executor: Finished task 0.0 in stage 120.0 (TID 408). 2296 bytes result sent to driver 20/08/03 12:50:49.587 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 120.0 (TID 410, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.587 Executor task launch worker for task 410 INFO Executor: Running task 2.0 in stage 120.0 (TID 410) 20/08/03 12:50:49.587 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 120.0 (TID 411, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.587 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 120.0 (TID 409) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.587 Executor task launch worker for task 411 INFO Executor: Running task 3.0 in stage 120.0 (TID 411) 20/08/03 12:50:49.587 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 120.0 (TID 408) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.589 Executor task launch worker for task 410 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.589 Executor task launch worker for task 410 INFO Executor: Finished task 2.0 in stage 120.0 (TID 410). 2296 bytes result sent to driver 20/08/03 12:50:49.589 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 120.0 (TID 410) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.590 Executor task launch worker for task 411 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.591 Executor task launch worker for task 411 INFO Executor: Finished task 3.0 in stage 120.0 (TID 411). 2296 bytes result sent to driver 20/08/03 12:50:49.591 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 120.0 (TID 411) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.591 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 120.0, whose tasks have all completed, from pool 20/08/03 12:50:49.592 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 120 (treeAggregate at RDDLossFunction.scala:61) finished in 0.012 s 20/08/03 12:50:49.592 dag-scheduler-event-loop INFO DAGScheduler: Job 120 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.592 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 120: Stage finished 20/08/03 12:50:49.592 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 120 finished: treeAggregate at RDDLossFunction.scala:61, took 0.014240 s 20/08/03 12:50:49.592 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(236) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.593 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.593 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.701447 (rel: 1.01e-05) 0.00522456 20/08/03 12:50:49.593 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_236_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.594 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_238 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.596 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_238_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.596 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_238_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.596 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 238 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.604 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.605 dag-scheduler-event-loop INFO DAGScheduler: Got job 121 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.605 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 121 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.605 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.605 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.605 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 121 (MapPartitionsRDD[441] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.607 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_239 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.608 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_239_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.608 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_239_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.608 dag-scheduler-event-loop INFO SparkContext: Created broadcast 239 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.608 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 121 (MapPartitionsRDD[441] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.608 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 121.0 with 4 tasks 20/08/03 12:50:49.609 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 121.0 (TID 412, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.609 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 121.0 (TID 413, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.609 Executor task launch worker for task 413 INFO Executor: Running task 1.0 in stage 121.0 (TID 413) 20/08/03 12:50:49.609 Executor task launch worker for task 412 INFO Executor: Running task 0.0 in stage 121.0 (TID 412) 20/08/03 12:50:49.611 Executor task launch worker for task 413 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.611 Executor task launch worker for task 412 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.611 Executor task launch worker for task 413 INFO Executor: Finished task 1.0 in stage 121.0 (TID 413). 2296 bytes result sent to driver 20/08/03 12:50:49.612 Executor task launch worker for task 412 INFO Executor: Finished task 0.0 in stage 121.0 (TID 412). 2296 bytes result sent to driver 20/08/03 12:50:49.612 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 121.0 (TID 414, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.612 Executor task launch worker for task 414 INFO Executor: Running task 2.0 in stage 121.0 (TID 414) 20/08/03 12:50:49.612 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 121.0 (TID 415, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.612 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 121.0 (TID 413) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.612 Executor task launch worker for task 415 INFO Executor: Running task 3.0 in stage 121.0 (TID 415) 20/08/03 12:50:49.612 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 121.0 (TID 412) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.613 Executor task launch worker for task 414 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.614 Executor task launch worker for task 414 INFO Executor: Finished task 2.0 in stage 121.0 (TID 414). 2296 bytes result sent to driver 20/08/03 12:50:49.614 Executor task launch worker for task 415 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.614 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 121.0 (TID 414) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.614 Executor task launch worker for task 415 INFO Executor: Finished task 3.0 in stage 121.0 (TID 415). 2296 bytes result sent to driver 20/08/03 12:50:49.614 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 121.0 (TID 415) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.614 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 121.0, whose tasks have all completed, from pool 20/08/03 12:50:49.615 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 121 (treeAggregate at RDDLossFunction.scala:61) finished in 0.008 s 20/08/03 12:50:49.615 dag-scheduler-event-loop INFO DAGScheduler: Job 121 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.615 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 121: Stage finished 20/08/03 12:50:49.615 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 121 finished: treeAggregate at RDDLossFunction.scala:61, took 0.010900 s 20/08/03 12:50:49.615 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(238) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.616 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.616 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_238_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.616 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.701446 (rel: 1.42e-06) 0.000397648 20/08/03 12:50:49.617 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_240 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.618 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_240_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.618 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_240_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.618 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 240 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.624 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.625 dag-scheduler-event-loop INFO DAGScheduler: Got job 122 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.625 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 122 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.625 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.625 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.626 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 122 (MapPartitionsRDD[442] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.627 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_241 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.628 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_241_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.628 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_241_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.628 dag-scheduler-event-loop INFO SparkContext: Created broadcast 241 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.629 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 122 (MapPartitionsRDD[442] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.629 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 122.0 with 4 tasks 20/08/03 12:50:49.629 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 122.0 (TID 416, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.630 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 122.0 (TID 417, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.630 Executor task launch worker for task 417 INFO Executor: Running task 1.0 in stage 122.0 (TID 417) 20/08/03 12:50:49.630 Executor task launch worker for task 416 INFO Executor: Running task 0.0 in stage 122.0 (TID 416) 20/08/03 12:50:49.632 Executor task launch worker for task 417 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.632 Executor task launch worker for task 416 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.632 Executor task launch worker for task 417 INFO Executor: Finished task 1.0 in stage 122.0 (TID 417). 2296 bytes result sent to driver 20/08/03 12:50:49.633 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 122.0 (TID 418, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.633 Executor task launch worker for task 416 INFO Executor: Finished task 0.0 in stage 122.0 (TID 416). 2296 bytes result sent to driver 20/08/03 12:50:49.633 Executor task launch worker for task 418 INFO Executor: Running task 2.0 in stage 122.0 (TID 418) 20/08/03 12:50:49.633 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 122.0 (TID 417) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.633 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 122.0 (TID 419, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.633 Executor task launch worker for task 419 INFO Executor: Running task 3.0 in stage 122.0 (TID 419) 20/08/03 12:50:49.633 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 122.0 (TID 416) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.634 Executor task launch worker for task 418 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.635 Executor task launch worker for task 418 INFO Executor: Finished task 2.0 in stage 122.0 (TID 418). 2296 bytes result sent to driver 20/08/03 12:50:49.635 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 122.0 (TID 418) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.635 Executor task launch worker for task 419 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.635 Executor task launch worker for task 419 INFO Executor: Finished task 3.0 in stage 122.0 (TID 419). 2296 bytes result sent to driver 20/08/03 12:50:49.636 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 122.0 (TID 419) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.636 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 122.0, whose tasks have all completed, from pool 20/08/03 12:50:49.636 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 122 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:49.636 dag-scheduler-event-loop INFO DAGScheduler: Job 122 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.636 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 122: Stage finished 20/08/03 12:50:49.636 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 122 finished: treeAggregate at RDDLossFunction.scala:61, took 0.011779 s 20/08/03 12:50:49.637 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(240) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.637 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.637 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.701446 (rel: 6.15e-08) 0.000190140 20/08/03 12:50:49.637 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_240_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.638 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_242 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.639 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_242_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.639 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_242_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.639 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 242 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.646 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.646 dag-scheduler-event-loop INFO DAGScheduler: Got job 123 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.646 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 123 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.646 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.646 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.646 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 123 (MapPartitionsRDD[443] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.648 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_243 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.649 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_243_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.649 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_243_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.649 dag-scheduler-event-loop INFO SparkContext: Created broadcast 243 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.649 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 123 (MapPartitionsRDD[443] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.649 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 123.0 with 4 tasks 20/08/03 12:50:49.650 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 123.0 (TID 420, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.650 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 123.0 (TID 421, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.650 Executor task launch worker for task 420 INFO Executor: Running task 0.0 in stage 123.0 (TID 420) 20/08/03 12:50:49.650 Executor task launch worker for task 421 INFO Executor: Running task 1.0 in stage 123.0 (TID 421) 20/08/03 12:50:49.652 Executor task launch worker for task 421 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.653 Executor task launch worker for task 421 INFO Executor: Finished task 1.0 in stage 123.0 (TID 421). 2296 bytes result sent to driver 20/08/03 12:50:49.653 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 123.0 (TID 422, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.653 Executor task launch worker for task 422 INFO Executor: Running task 2.0 in stage 123.0 (TID 422) 20/08/03 12:50:49.653 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 123.0 (TID 421) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.655 Executor task launch worker for task 422 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.655 Executor task launch worker for task 422 INFO Executor: Finished task 2.0 in stage 123.0 (TID 422). 2296 bytes result sent to driver 20/08/03 12:50:49.655 Executor task launch worker for task 420 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.655 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 123.0 (TID 423, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.655 Executor task launch worker for task 423 INFO Executor: Running task 3.0 in stage 123.0 (TID 423) 20/08/03 12:50:49.656 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 123.0 (TID 422) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.656 Executor task launch worker for task 420 INFO Executor: Finished task 0.0 in stage 123.0 (TID 420). 2296 bytes result sent to driver 20/08/03 12:50:49.656 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 123.0 (TID 420) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.657 Executor task launch worker for task 423 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.657 Executor task launch worker for task 423 INFO Executor: Finished task 3.0 in stage 123.0 (TID 423). 2296 bytes result sent to driver 20/08/03 12:50:49.657 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 123.0 (TID 423) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.657 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 123.0, whose tasks have all completed, from pool 20/08/03 12:50:49.658 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 123 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:49.658 dag-scheduler-event-loop INFO DAGScheduler: Job 123 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.658 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 123: Stage finished 20/08/03 12:50:49.658 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 123 finished: treeAggregate at RDDLossFunction.scala:61, took 0.012215 s 20/08/03 12:50:49.658 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(242) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.659 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.659 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_242_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.659 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.701446 (rel: 1.48e-08) 2.39652e-05 20/08/03 12:50:49.659 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_244 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.660 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_244_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.660 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_244_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.661 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 244 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.667 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.667 dag-scheduler-event-loop INFO DAGScheduler: Got job 124 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.667 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 124 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.667 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.668 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.668 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 124 (MapPartitionsRDD[444] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.669 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_245 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.670 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_245_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.671 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_245_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.671 dag-scheduler-event-loop INFO SparkContext: Created broadcast 245 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.671 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 124 (MapPartitionsRDD[444] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.671 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 124.0 with 4 tasks 20/08/03 12:50:49.672 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 124.0 (TID 424, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.672 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 124.0 (TID 425, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.672 Executor task launch worker for task 425 INFO Executor: Running task 1.0 in stage 124.0 (TID 425) 20/08/03 12:50:49.672 Executor task launch worker for task 424 INFO Executor: Running task 0.0 in stage 124.0 (TID 424) 20/08/03 12:50:49.673 Executor task launch worker for task 425 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.674 Executor task launch worker for task 424 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.674 Executor task launch worker for task 425 INFO Executor: Finished task 1.0 in stage 124.0 (TID 425). 2296 bytes result sent to driver 20/08/03 12:50:49.674 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 124.0 (TID 426, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.674 Executor task launch worker for task 424 INFO Executor: Finished task 0.0 in stage 124.0 (TID 424). 2296 bytes result sent to driver 20/08/03 12:50:49.674 Executor task launch worker for task 426 INFO Executor: Running task 2.0 in stage 124.0 (TID 426) 20/08/03 12:50:49.674 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 124.0 (TID 425) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.675 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 124.0 (TID 427, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.675 Executor task launch worker for task 427 INFO Executor: Running task 3.0 in stage 124.0 (TID 427) 20/08/03 12:50:49.675 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 124.0 (TID 424) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.676 Executor task launch worker for task 426 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.676 Executor task launch worker for task 426 INFO Executor: Finished task 2.0 in stage 124.0 (TID 426). 2296 bytes result sent to driver 20/08/03 12:50:49.676 Executor task launch worker for task 427 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.676 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 124.0 (TID 426) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.676 Executor task launch worker for task 427 INFO Executor: Finished task 3.0 in stage 124.0 (TID 427). 2296 bytes result sent to driver 20/08/03 12:50:49.677 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 124.0 (TID 427) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.677 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 124.0, whose tasks have all completed, from pool 20/08/03 12:50:49.677 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 124 (treeAggregate at RDDLossFunction.scala:61) finished in 0.009 s 20/08/03 12:50:49.677 dag-scheduler-event-loop INFO DAGScheduler: Job 124 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.677 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 124: Stage finished 20/08/03 12:50:49.677 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 124 finished: treeAggregate at RDDLossFunction.scala:61, took 0.010008 s 20/08/03 12:50:49.678 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(244) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.678 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_244_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.678 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_246 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.679 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_246_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.679 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_246_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.680 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 246 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.686 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.687 dag-scheduler-event-loop INFO DAGScheduler: Got job 125 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.687 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 125 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.687 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.687 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.687 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 125 (MapPartitionsRDD[445] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.688 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_247 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.690 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_247_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.690 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_247_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.690 dag-scheduler-event-loop INFO SparkContext: Created broadcast 247 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.690 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 125 (MapPartitionsRDD[445] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.690 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 125.0 with 4 tasks 20/08/03 12:50:49.691 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 125.0 (TID 428, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.691 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 125.0 (TID 429, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.692 Executor task launch worker for task 428 INFO Executor: Running task 0.0 in stage 125.0 (TID 428) 20/08/03 12:50:49.692 Executor task launch worker for task 429 INFO Executor: Running task 1.0 in stage 125.0 (TID 429) 20/08/03 12:50:49.694 Executor task launch worker for task 428 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.694 Executor task launch worker for task 429 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.694 Executor task launch worker for task 428 INFO Executor: Finished task 0.0 in stage 125.0 (TID 428). 2296 bytes result sent to driver 20/08/03 12:50:49.694 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 125.0 (TID 430, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.694 Executor task launch worker for task 429 INFO Executor: Finished task 1.0 in stage 125.0 (TID 429). 2296 bytes result sent to driver 20/08/03 12:50:49.694 Executor task launch worker for task 430 INFO Executor: Running task 2.0 in stage 125.0 (TID 430) 20/08/03 12:50:49.695 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 125.0 (TID 431, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.695 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 125.0 (TID 428) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.695 Executor task launch worker for task 431 INFO Executor: Running task 3.0 in stage 125.0 (TID 431) 20/08/03 12:50:49.695 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 125.0 (TID 429) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.696 Executor task launch worker for task 430 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.696 Executor task launch worker for task 431 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.696 Executor task launch worker for task 430 INFO Executor: Finished task 2.0 in stage 125.0 (TID 430). 2296 bytes result sent to driver 20/08/03 12:50:49.696 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 125.0 (TID 430) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.696 Executor task launch worker for task 431 INFO Executor: Finished task 3.0 in stage 125.0 (TID 431). 2296 bytes result sent to driver 20/08/03 12:50:49.697 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 125.0 (TID 431) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.697 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 125.0, whose tasks have all completed, from pool 20/08/03 12:50:49.697 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 125 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:49.698 dag-scheduler-event-loop INFO DAGScheduler: Job 125 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.698 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 125: Stage finished 20/08/03 12:50:49.698 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 125 finished: treeAggregate at RDDLossFunction.scala:61, took 0.011412 s 20/08/03 12:50:49.698 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(246) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.699 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO StrongWolfeLineSearch: Line search t: 0.27488133457628927 fval: 0.7014458400271684 rhs: 0.7014458400400051 cdd: 1.6483328007856755E-16 20/08/03 12:50:49.699 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 0.2749 20/08/03 12:50:49.699 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.701446 (rel: 1.83e-11) 1.13837e-05 20/08/03 12:50:49.699 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_246_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.699 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_248 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.701 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_248_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.701 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_248_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.701 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 248 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.708 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.709 dag-scheduler-event-loop INFO DAGScheduler: Got job 126 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.709 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 126 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.709 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.709 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.709 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 126 (MapPartitionsRDD[446] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.711 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_249 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.713 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_249_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.713 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_249_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.713 dag-scheduler-event-loop INFO SparkContext: Created broadcast 249 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.713 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 126 (MapPartitionsRDD[446] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.713 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 126.0 with 4 tasks 20/08/03 12:50:49.715 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 126.0 (TID 432, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.715 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 126.0 (TID 433, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.715 Executor task launch worker for task 433 INFO Executor: Running task 1.0 in stage 126.0 (TID 433) 20/08/03 12:50:49.715 Executor task launch worker for task 432 INFO Executor: Running task 0.0 in stage 126.0 (TID 432) 20/08/03 12:50:49.718 Executor task launch worker for task 432 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.718 Executor task launch worker for task 433 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.718 Executor task launch worker for task 432 INFO Executor: Finished task 0.0 in stage 126.0 (TID 432). 2296 bytes result sent to driver 20/08/03 12:50:49.718 Executor task launch worker for task 433 INFO Executor: Finished task 1.0 in stage 126.0 (TID 433). 2339 bytes result sent to driver 20/08/03 12:50:49.719 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 126.0 (TID 434, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.719 Executor task launch worker for task 434 INFO Executor: Running task 2.0 in stage 126.0 (TID 434) 20/08/03 12:50:49.719 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 126.0 (TID 435, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.719 Executor task launch worker for task 435 INFO Executor: Running task 3.0 in stage 126.0 (TID 435) 20/08/03 12:50:49.719 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 126.0 (TID 432) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.719 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 126.0 (TID 433) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.721 Executor task launch worker for task 435 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.721 Executor task launch worker for task 434 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.721 Executor task launch worker for task 435 INFO Executor: Finished task 3.0 in stage 126.0 (TID 435). 2296 bytes result sent to driver 20/08/03 12:50:49.721 Executor task launch worker for task 434 INFO Executor: Finished task 2.0 in stage 126.0 (TID 434). 2296 bytes result sent to driver 20/08/03 12:50:49.721 task-result-getter-2 INFO TaskSetManager: Finished task 3.0 in stage 126.0 (TID 435) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.722 task-result-getter-3 INFO TaskSetManager: Finished task 2.0 in stage 126.0 (TID 434) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.722 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 126.0, whose tasks have all completed, from pool 20/08/03 12:50:49.722 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 126 (treeAggregate at RDDLossFunction.scala:61) finished in 0.012 s 20/08/03 12:50:49.722 dag-scheduler-event-loop INFO DAGScheduler: Job 126 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.722 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 126: Stage finished 20/08/03 12:50:49.722 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 126 finished: treeAggregate at RDDLossFunction.scala:61, took 0.013945 s 20/08/03 12:50:49.723 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(248) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.723 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.723 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.701446 (rel: 7.35e-12) 8.92229e-07 20/08/03 12:50:49.723 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_248_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.724 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_250 stored as values in memory (estimated size 128.0 B, free 2.1 GiB) 20/08/03 12:50:49.725 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MemoryStore: Block broadcast_250_piece0 stored as bytes in memory (estimated size 210.0 B, free 2.1 GiB) 20/08/03 12:50:49.725 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_250_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.725 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Created broadcast 250 from broadcast at RDDLossFunction.scala:57 20/08/03 12:50:49.732 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: treeAggregate at RDDLossFunction.scala:61 20/08/03 12:50:49.733 dag-scheduler-event-loop INFO DAGScheduler: Got job 127 (treeAggregate at RDDLossFunction.scala:61) with 4 output partitions 20/08/03 12:50:49.733 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 127 (treeAggregate at RDDLossFunction.scala:61) 20/08/03 12:50:49.733 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.733 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.733 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 127 (MapPartitionsRDD[447] at treeAggregate at RDDLossFunction.scala:61), which has no missing parents 20/08/03 12:50:49.735 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_251 stored as values in memory (estimated size 26.7 KiB, free 2.1 GiB) 20/08/03 12:50:49.736 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_251_piece0 stored as bytes in memory (estimated size 10.3 KiB, free 2.1 GiB) 20/08/03 12:50:49.736 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_251_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.3 KiB, free: 2.1 GiB) 20/08/03 12:50:49.736 dag-scheduler-event-loop INFO SparkContext: Created broadcast 251 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.737 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 127 (MapPartitionsRDD[447] at treeAggregate at RDDLossFunction.scala:61) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.737 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 127.0 with 4 tasks 20/08/03 12:50:49.738 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 127.0 (TID 436, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.738 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 127.0 (TID 437, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.738 Executor task launch worker for task 436 INFO Executor: Running task 0.0 in stage 127.0 (TID 436) 20/08/03 12:50:49.738 Executor task launch worker for task 437 INFO Executor: Running task 1.0 in stage 127.0 (TID 437) 20/08/03 12:50:49.741 Executor task launch worker for task 437 INFO BlockManager: Found block rdd_389_1 locally 20/08/03 12:50:49.741 Executor task launch worker for task 436 INFO BlockManager: Found block rdd_389_0 locally 20/08/03 12:50:49.742 Executor task launch worker for task 436 INFO Executor: Finished task 0.0 in stage 127.0 (TID 436). 2339 bytes result sent to driver 20/08/03 12:50:49.742 Executor task launch worker for task 437 INFO Executor: Finished task 1.0 in stage 127.0 (TID 437). 2296 bytes result sent to driver 20/08/03 12:50:49.742 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 127.0 (TID 438, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.742 Executor task launch worker for task 438 INFO Executor: Running task 2.0 in stage 127.0 (TID 438) 20/08/03 12:50:49.742 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 127.0 (TID 439, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.742 Executor task launch worker for task 439 INFO Executor: Running task 3.0 in stage 127.0 (TID 439) 20/08/03 12:50:49.742 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 127.0 (TID 436) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.742 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 127.0 (TID 437) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.743 Executor task launch worker for task 438 INFO BlockManager: Found block rdd_389_2 locally 20/08/03 12:50:49.743 Executor task launch worker for task 439 INFO BlockManager: Found block rdd_389_3 locally 20/08/03 12:50:49.743 Executor task launch worker for task 438 INFO Executor: Finished task 2.0 in stage 127.0 (TID 438). 2296 bytes result sent to driver 20/08/03 12:50:49.744 Executor task launch worker for task 439 INFO Executor: Finished task 3.0 in stage 127.0 (TID 439). 2296 bytes result sent to driver 20/08/03 12:50:49.744 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 127.0 (TID 438) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.744 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 127.0 (TID 439) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.744 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 127.0, whose tasks have all completed, from pool 20/08/03 12:50:49.744 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 127 (treeAggregate at RDDLossFunction.scala:61) finished in 0.010 s 20/08/03 12:50:49.744 dag-scheduler-event-loop INFO DAGScheduler: Job 127 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.745 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 127: Stage finished 20/08/03 12:50:49.745 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 127 finished: treeAggregate at RDDLossFunction.scala:61, took 0.012513 s 20/08/03 12:50:49.745 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(250) (from destroy at RDDLossFunction.scala:68) 20/08/03 12:50:49.746 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Step Size: 1.000 20/08/03 12:50:49.746 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_250_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 210.0 B, free: 2.1 GiB) 20/08/03 12:50:49.746 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Val and Grad Norm: 0.701446 (rel: 6.32e-14) 1.37037e-08 20/08/03 12:50:49.746 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LBFGS: Converged because gradient converged 20/08/03 12:50:49.746 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO TorrentBroadcast: Destroying Broadcast(147) (from destroy at LogisticRegression.scala:804) 20/08/03 12:50:49.746 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MapPartitionsRDD: Removing RDD 389 from persistence list 20/08/03 12:50:49.746 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Removed broadcast_147_piece0 on amp-jenkins-worker-06.amp:34426 in memory (size: 85.0 B, free: 2.1 GiB) 20/08/03 12:50:49.747 block-manager-slave-async-thread-pool-21 INFO BlockManager: Removing RDD 389 20/08/03 12:50:49.812 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO Instrumentation: [d2b78bff] training finished 20/08/03 12:50:49.872 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 15.161184 ms 20/08/03 12:50:49.883 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: collect at LogisticRegressionSuite.scala:353 20/08/03 12:50:49.883 dag-scheduler-event-loop INFO DAGScheduler: Got job 128 (collect at LogisticRegressionSuite.scala:353) with 4 output partitions 20/08/03 12:50:49.884 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 128 (collect at LogisticRegressionSuite.scala:353) 20/08/03 12:50:49.884 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.884 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.884 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 128 (MapPartitionsRDD[456] at collect at LogisticRegressionSuite.scala:353), which has no missing parents 20/08/03 12:50:49.885 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_252 stored as values in memory (estimated size 25.9 KiB, free 2.1 GiB) 20/08/03 12:50:49.887 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_252_piece0 stored as bytes in memory (estimated size 10.9 KiB, free 2.1 GiB) 20/08/03 12:50:49.887 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_252_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.9 KiB, free: 2.1 GiB) 20/08/03 12:50:49.887 dag-scheduler-event-loop INFO SparkContext: Created broadcast 252 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.887 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 128 (MapPartitionsRDD[456] at collect at LogisticRegressionSuite.scala:353) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.887 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 128.0 with 4 tasks 20/08/03 12:50:49.888 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 128.0 (TID 440, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.888 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 128.0 (TID 441, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.888 Executor task launch worker for task 441 INFO Executor: Running task 1.0 in stage 128.0 (TID 441) 20/08/03 12:50:49.888 Executor task launch worker for task 440 INFO Executor: Running task 0.0 in stage 128.0 (TID 440) 20/08/03 12:50:49.909 Executor task launch worker for task 441 INFO Executor: Finished task 1.0 in stage 128.0 (TID 441). 1401 bytes result sent to driver 20/08/03 12:50:49.909 Executor task launch worker for task 440 INFO Executor: Finished task 0.0 in stage 128.0 (TID 440). 1407 bytes result sent to driver 20/08/03 12:50:49.909 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 128.0 (TID 442, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.909 Executor task launch worker for task 442 INFO Executor: Running task 2.0 in stage 128.0 (TID 442) 20/08/03 12:50:49.910 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 128.0 (TID 443, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.910 Executor task launch worker for task 443 INFO Executor: Running task 3.0 in stage 128.0 (TID 443) 20/08/03 12:50:49.910 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 128.0 (TID 441) in 22 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.910 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 128.0 (TID 440) in 22 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.913 Executor task launch worker for task 442 INFO Executor: Finished task 2.0 in stage 128.0 (TID 442). 1425 bytes result sent to driver 20/08/03 12:50:49.913 Executor task launch worker for task 443 INFO Executor: Finished task 3.0 in stage 128.0 (TID 443). 1428 bytes result sent to driver 20/08/03 12:50:49.914 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 128.0 (TID 442) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.914 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 128.0 (TID 443) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.914 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 128.0, whose tasks have all completed, from pool 20/08/03 12:50:49.914 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 128 (collect at LogisticRegressionSuite.scala:353) finished in 0.030 s 20/08/03 12:50:49.915 dag-scheduler-event-loop INFO DAGScheduler: Job 128 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.915 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 128: Stage finished 20/08/03 12:50:49.915 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 128 finished: collect at LogisticRegressionSuite.scala:353, took 0.031635 s 20/08/03 12:50:49.962 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: collect at MLTest.scala:116 20/08/03 12:50:49.963 dag-scheduler-event-loop INFO DAGScheduler: Got job 129 (collect at MLTest.scala:116) with 4 output partitions 20/08/03 12:50:49.963 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 129 (collect at MLTest.scala:116) 20/08/03 12:50:49.963 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:49.963 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:49.963 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 129 (MapPartitionsRDD[459] at collect at MLTest.scala:116), which has no missing parents 20/08/03 12:50:49.965 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_253 stored as values in memory (estimated size 15.2 KiB, free 2.1 GiB) 20/08/03 12:50:49.967 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_253_piece0 stored as bytes in memory (estimated size 5.8 KiB, free 2.1 GiB) 20/08/03 12:50:49.967 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_253_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.8 KiB, free: 2.1 GiB) 20/08/03 12:50:49.967 dag-scheduler-event-loop INFO SparkContext: Created broadcast 253 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:49.968 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 129 (MapPartitionsRDD[459] at collect at MLTest.scala:116) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:49.968 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 129.0 with 4 tasks 20/08/03 12:50:49.968 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 129.0 (TID 444, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.969 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 129.0 (TID 445, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.969 Executor task launch worker for task 445 INFO Executor: Running task 1.0 in stage 129.0 (TID 445) 20/08/03 12:50:49.969 Executor task launch worker for task 444 INFO Executor: Running task 0.0 in stage 129.0 (TID 444) 20/08/03 12:50:49.975 Executor task launch worker for task 445 INFO Executor: Finished task 1.0 in stage 129.0 (TID 445). 1970 bytes result sent to driver 20/08/03 12:50:49.975 Executor task launch worker for task 444 INFO Executor: Finished task 0.0 in stage 129.0 (TID 444). 1955 bytes result sent to driver 20/08/03 12:50:49.975 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 129.0 (TID 446, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.976 Executor task launch worker for task 446 INFO Executor: Running task 2.0 in stage 129.0 (TID 446) 20/08/03 12:50:49.976 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 129.0 (TID 447, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:49.976 Executor task launch worker for task 447 INFO Executor: Running task 3.0 in stage 129.0 (TID 447) 20/08/03 12:50:49.976 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 129.0 (TID 445) in 7 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:49.976 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 129.0 (TID 444) in 8 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:49.977 Executor task launch worker for task 446 INFO Executor: Finished task 2.0 in stage 129.0 (TID 446). 1876 bytes result sent to driver 20/08/03 12:50:49.977 Executor task launch worker for task 447 INFO Executor: Finished task 3.0 in stage 129.0 (TID 447). 1856 bytes result sent to driver 20/08/03 12:50:49.977 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 129.0 (TID 446) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:49.978 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 129.0 (TID 447) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:49.978 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 129.0, whose tasks have all completed, from pool 20/08/03 12:50:49.978 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 129 (collect at MLTest.scala:116) finished in 0.014 s 20/08/03 12:50:49.978 dag-scheduler-event-loop INFO DAGScheduler: Job 129 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:49.978 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 129: Stage finished 20/08/03 12:50:49.978 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 129 finished: collect at MLTest.scala:116, took 0.015835 s 20/08/03 12:50:50.002 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: StartStream(ProcessingTimeTrigger(0),org.apache.spark.util.SystemClock@7b044ce8,Map(),null) 20/08/03 12:50:50.004 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Checkpoint root /home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1ed9595d-0d68-4e16-b5bd-7d389ec1c094 resolved to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1ed9595d-0d68-4e16-b5bd-7d389ec1c094. 20/08/03 12:50:50.009 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1ed9595d-0d68-4e16-b5bd-7d389ec1c094/metadata using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1ed9595d-0d68-4e16-b5bd-7d389ec1c094/.metadata.ffae3f63-8dc5-4822-a945-74a0d8c84a74.tmp 20/08/03 12:50:50.016 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1ed9595d-0d68-4e16-b5bd-7d389ec1c094/.metadata.ffae3f63-8dc5-4822-a945-74a0d8c84a74.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1ed9595d-0d68-4e16-b5bd-7d389ec1c094/metadata 20/08/03 12:50:50.023 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Starting [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea]. Use file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1ed9595d-0d68-4e16-b5bd-7d389ec1c094 to store the query checkpoint. 20/08/03 12:50:50.026 stream execution thread for [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea] INFO MicroBatchExecution: Reading table [org.apache.spark.sql.execution.streaming.MemoryStreamTable@7963e932] from DataSourceV2 named 'memory' [org.apache.spark.sql.execution.streaming.MemoryStreamTableProvider$@490dc5fd] 20/08/03 12:50:50.028 stream execution thread for [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea] INFO MicroBatchExecution: Starting new streaming query. 20/08/03 12:50:50.028 stream execution thread for [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea] INFO MicroBatchExecution: Stream started from {} 20/08/03 12:50:50.028 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: AddData to MemoryStream[_1#1001,_2#1002]: (0.0,[6.7885086340489185,3.4576551565453197]),(0.0,[5.056501247252454,2.574601118217243]),(0.0,[6.075652199068211,3.355341616104334]),(0.0,[5.166332778860912,2.448377481827237]),(0.0,[5.684896098313557,3.7046548957911765]),(0.0,[6.50731807165581,3.004047684427792]),(0.0,[7.010912781425465,2.7780024496013715]),(1.0,[4.8414027004203835,3.211158821420722]),(1.0,[5.436985752813093,3.296990772729685]),(2.0,[4.84644814192866,3.1968908375605634]),(0.0,[7.127365231828297,3.248100610649046]),(1.0,[6.241767847456931,3.7192174938869087]),(0.0,[5.613780653545803,3.020421477260098]),(0.0,[6.882841862374561,2.915254192504282]),(1.0,[5.699513929932456,2.247074740074896]),(0.0,[7.02192590749258,2.4627220581296783]),(1.0,[4.216237488061392,2.656699174446591]),(0.0,[3.7324966114897262,2.3462678397985552]),(1.0,[5.741978867831515,3.6187835154633294]),(0.0,[5.620150427156592,3.1692083626745053]),(0.0,[5.578107805726126,2.2863316707315384]),(0.0,[5.442698469962282,2.834758817537672]),(0.0,[6.767616345179818,3.0391562262408645]),(1.0,[4.925837281912354,3.8685534349585087]),(0.0,[6.791684731921275,2.595654759591007]),(0.0,[6.731122446118569,2.2151938610676063]),(0.0,[6.092251787382735,3.1648644467226514]),(0.0,[7.007311457155493,2.3945255550358033]),(1.0,[6.067332171369262,3.301578503282249]),(0.0,[5.404295990288988,3.291892487883425]),(0.0,[7.674839072483042,2.781183265589518]),(0.0,[4.2837121498611985,3.2254558585309234]),(1.0,[6.112789784401108,2.664233955695323]),(1.0,[5.840780678327939,2.8504603244268845]),(1.0,[5.389268793409797,3.0486766295260552]),(0.0,[7.206579880655595,2.5745696091376855]),(1.0,[6.334680742393493,2.989165223067271]),(0.0,[6.39220828708044,3.5916060870142212]),(1.0,[6.574935571365726,3.1536220523404945]),(0.0,[6.604522213141694,2.734798375932801]),(0.0,[6.508320460023237,2.946338492652808]),(1.0,[5.210534043458435,2.24447246587153]),(1.0,[4.301820482847355,2.8057320879830776]),(0.0,[5.504349176848336,3.165273107673573]),(0.0,[5.444799426865632,3.251448135769693]),(0.0,[5.473220085163719,3.076888207026409]),(0.0,[6.426387726380042,2.938545841808128]),(0.0,[5.909860769315067,3.6056829798513315]),(0.0,[6.475193522926937,3.807440177850298]),(0.0,[5.963848054893707,2.623550555268978]),(1.0,[6.009222629896281,3.4475575899620026]),(0.0,[7.094718114371827,2.4454009956067817]),(1.0,[4.264010706382008,3.65338684298196]),(0.0,[5.379585321764099,3.381481745785779]),(0.0,[7.146010317633724,3.5485733241937085]),(0.0,[5.683954590364761,2.0745476381163566]),(0.0,[6.2072015922395885,3.3432172748930773]),(0.0,[5.226174068792373,3.579701566179305]),(0.0,[7.316647506340967,3.0863730559231892]),(1.0,[6.915325522847731,2.803084035949653]),(0.0,[6.479630329902845,3.239007840935509]),(0.0,[5.454587096334029,3.2374754911843655]),(2.0,[6.391989224379249,3.434901891800874]),(1.0,[6.0886711934815265,3.1971117650588363]),(0.0,[7.684738047156347,3.421241896031828]),(0.0,[4.85493416785451,3.1087723328558345]),(0.0,[6.305063368354057,2.2060572623972567]),(0.0,[7.081748274709549,2.724685616028466]),(0.0,[6.584215742308165,2.9773965559705675]),(1.0,[6.612853131358183,2.8663569212164877]),(0.0,[5.756132728856622,2.7905503944752907]),(1.0,[6.135092550527577,2.5582480777704872]),(1.0,[5.301620953306969,4.223612451290534]),(1.0,[5.9682958432514,3.8618929524452805]),(2.0,[6.207939723169083,3.439309516984485]),(0.0,[6.627109164472448,2.7520560194737964]),(0.0,[3.9259860466490686,3.0983115739148284]),(0.0,[6.0020085765576665,2.6176816137758694]),(0.0,[4.062185373981144,3.76166658403873]),(1.0,[6.2962118879304585,3.764267301123263]),(1.0,[6.14908975112769,3.590411745294196]),(1.0,[6.567608573964195,2.9306232708439306]),(0.0,[6.201805185587324,3.4958144403354416]),(1.0,[7.319347971525034,3.648290394729337]),(1.0,[6.212234548496149,3.6633867263201436]),(0.0,[5.870287402238878,2.7075846255583254]),(0.0,[5.8287726136817,2.483308333263797]),(0.0,[5.819965967504458,3.9163413775672242]),(0.0,[6.982232083397012,2.503264912876912]),(0.0,[6.084806791595145,3.72518246843444]),(0.0,[5.92962672345617,2.484757315931346]),(1.0,[7.3168542953228615,3.229242214881765]),(0.0,[5.428611939474008,2.1676964414169606]),(1.0,[6.74298764459546,2.6034237914698872]),(0.0,[7.467271990105523,3.4604682039447336]),(0.0,[6.970465097926772,4.3162388355161205]),(0.0,[5.436435936556991,2.747513090695181]),(0.0,[7.143361618211275,2.6898992927088976]),(0.0,[5.568158116555633,2.316684365492898]),(0.0,[5.40320969646403,1.929404297480336]) 20/08/03 12:50:50.029 stream execution thread for [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea] INFO MicroBatchExecution: Streaming query made progress: { "id" : "4ea459d6-a026-4f8e-b652-cfc8808ea728", "runId" : "7f6c6bec-cd05-430e-a24c-7e163a8f7dea", "name" : null, "timestamp" : "2020-08-03T19:50:50.028Z", "batchId" : 0, "numInputRows" : 0, "processedRowsPerSecond" : 0.0, "durationMs" : { "latestOffset" : 0, "triggerExecution" : 0 }, "stateOperators" : [ ], "sources" : [ { "description" : "MemoryStream[_1#1001,_2#1002]", "startOffset" : null, "endOffset" : null, "numInputRows" : 0, "processedRowsPerSecond" : 0.0 } ], "sink" : { "description" : "MemorySink", "numOutputRows" : -1 } } 20/08/03 12:50:50.032 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: CheckAnswerByFunc 20/08/03 12:50:50.044 stream execution thread for [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea] INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1ed9595d-0d68-4e16-b5bd-7d389ec1c094/offsets/0 using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1ed9595d-0d68-4e16-b5bd-7d389ec1c094/offsets/.0.7f6e1154-cffd-4323-a25f-8620c0b793f6.tmp 20/08/03 12:50:50.054 stream execution thread for [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea] INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1ed9595d-0d68-4e16-b5bd-7d389ec1c094/offsets/.0.7f6e1154-cffd-4323-a25f-8620c0b793f6.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1ed9595d-0d68-4e16-b5bd-7d389ec1c094/offsets/0 20/08/03 12:50:50.054 stream execution thread for [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea] INFO MicroBatchExecution: Committed offsets for batch 0. Metadata OffsetSeqMetadata(0,1596484250039,Map(spark.sql.streaming.stateStore.providerClass -> org.apache.spark.sql.execution.streaming.state.HDFSBackedStateStoreProvider, spark.sql.streaming.join.stateFormatVersion -> 2, spark.sql.streaming.flatMapGroupsWithState.stateFormatVersion -> 2, spark.sql.streaming.multipleWatermarkPolicy -> min, spark.sql.streaming.aggregation.stateFormatVersion -> 2, spark.sql.shuffle.partitions -> 5)) 20/08/03 12:50:50.097 stream execution thread for [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea] INFO WriteToDataSourceV2Exec: Start processing data source write support: org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@36990255. The input RDD has 1 partitions. 20/08/03 12:50:50.097 stream execution thread for [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea] INFO SparkContext: Starting job: testStream$ at LogisticRegressionSuite.scala:39 20/08/03 12:50:50.098 dag-scheduler-event-loop INFO DAGScheduler: Got job 130 (testStream$ at LogisticRegressionSuite.scala:39) with 1 output partitions 20/08/03 12:50:50.098 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 130 (testStream$ at LogisticRegressionSuite.scala:39) 20/08/03 12:50:50.098 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:50.098 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:50.098 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 130 (MapPartitionsRDD[462] at testStream$ at LogisticRegressionSuite.scala:39), which has no missing parents 20/08/03 12:50:50.100 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_254 stored as values in memory (estimated size 22.5 KiB, free 2.1 GiB) 20/08/03 12:50:50.101 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_254_piece0 stored as bytes in memory (estimated size 9.9 KiB, free 2.1 GiB) 20/08/03 12:50:50.101 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_254_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 9.9 KiB, free: 2.1 GiB) 20/08/03 12:50:50.101 dag-scheduler-event-loop INFO SparkContext: Created broadcast 254 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:50.102 dag-scheduler-event-loop INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 130 (MapPartitionsRDD[462] at testStream$ at LogisticRegressionSuite.scala:39) (first 15 tasks are for partitions Vector(0)) 20/08/03 12:50:50.102 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 130.0 with 1 tasks 20/08/03 12:50:50.102 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 130.0 (TID 448, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 18876 bytes) 20/08/03 12:50:50.103 Executor task launch worker for task 448 INFO Executor: Running task 0.0 in stage 130.0 (TID 448) 20/08/03 12:50:50.115 Executor task launch worker for task 448 INFO DataWritingSparkTask: Commit authorized for partition 0 (task 448, attempt 0, stage 130.0) 20/08/03 12:50:50.116 Executor task launch worker for task 448 INFO DataWritingSparkTask: Committed partition 0 (task 448, attempt 0, stage 130.0) 20/08/03 12:50:50.117 Executor task launch worker for task 448 INFO Executor: Finished task 0.0 in stage 130.0 (TID 448). 6472 bytes result sent to driver 20/08/03 12:50:50.117 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 130.0 (TID 448) in 15 ms on amp-jenkins-worker-06.amp (executor driver) (1/1) 20/08/03 12:50:50.117 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 130.0, whose tasks have all completed, from pool 20/08/03 12:50:50.118 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 130 (testStream$ at LogisticRegressionSuite.scala:39) finished in 0.019 s 20/08/03 12:50:50.118 dag-scheduler-event-loop INFO DAGScheduler: Job 130 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:50.118 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 130: Stage finished 20/08/03 12:50:50.118 stream execution thread for [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea] INFO DAGScheduler: Job 130 finished: testStream$ at LogisticRegressionSuite.scala:39, took 0.020756 s 20/08/03 12:50:50.118 stream execution thread for [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea] INFO WriteToDataSourceV2Exec: Data source write support org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@36990255 is committing. 20/08/03 12:50:50.119 stream execution thread for [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea] INFO WriteToDataSourceV2Exec: Data source write support org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@36990255 committed. 20/08/03 12:50:50.124 stream execution thread for [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea] INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1ed9595d-0d68-4e16-b5bd-7d389ec1c094/commits/0 using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1ed9595d-0d68-4e16-b5bd-7d389ec1c094/commits/.0.d6bf329a-929c-47ec-8915-7c32343bee6b.tmp 20/08/03 12:50:50.133 stream execution thread for [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea] INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1ed9595d-0d68-4e16-b5bd-7d389ec1c094/commits/.0.d6bf329a-929c-47ec-8915-7c32343bee6b.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1ed9595d-0d68-4e16-b5bd-7d389ec1c094/commits/0 20/08/03 12:50:50.134 stream execution thread for [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea] INFO MicroBatchExecution: Streaming query made progress: { "id" : "4ea459d6-a026-4f8e-b652-cfc8808ea728", "runId" : "7f6c6bec-cd05-430e-a24c-7e163a8f7dea", "name" : null, "timestamp" : "2020-08-03T19:50:50.039Z", "batchId" : 0, "numInputRows" : 100, "inputRowsPerSecond" : 9090.909090909092, "processedRowsPerSecond" : 1063.8297872340424, "durationMs" : { "addBatch" : 38, "getBatch" : 0, "latestOffset" : 0, "queryPlanning" : 26, "triggerExecution" : 94, "walCommit" : 15 }, "stateOperators" : [ ], "sources" : [ { "description" : "MemoryStream[_1#1001,_2#1002]", "startOffset" : null, "endOffset" : 0, "numInputRows" : 100, "inputRowsPerSecond" : 9090.909090909092, "processedRowsPerSecond" : 1063.8297872340424 } ], "sink" : { "description" : "MemorySink", "numOutputRows" : 100 } } 20/08/03 12:50:50.146 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Asked to cancel job group 7f6c6bec-cd05-430e-a24c-7e163a8f7dea 20/08/03 12:50:50.147 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Asked to cancel job group 7f6c6bec-cd05-430e-a24c-7e163a8f7dea 20/08/03 12:50:50.147 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Query [id = 4ea459d6-a026-4f8e-b652-cfc8808ea728, runId = 7f6c6bec-cd05-430e-a24c-7e163a8f7dea] was stopped 20/08/03 12:50:50.199 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: collect at MLTest.scala:133 20/08/03 12:50:50.200 dag-scheduler-event-loop INFO DAGScheduler: Got job 131 (collect at MLTest.scala:133) with 4 output partitions 20/08/03 12:50:50.201 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 131 (collect at MLTest.scala:133) 20/08/03 12:50:50.201 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:50.201 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:50.201 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 131 (MapPartitionsRDD[465] at collect at MLTest.scala:133), which has no missing parents 20/08/03 12:50:50.202 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_255 stored as values in memory (estimated size 26.0 KiB, free 2.1 GiB) 20/08/03 12:50:50.204 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_255_piece0 stored as bytes in memory (estimated size 10.9 KiB, free 2.1 GiB) 20/08/03 12:50:50.205 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_255_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.9 KiB, free: 2.1 GiB) 20/08/03 12:50:50.205 dag-scheduler-event-loop INFO SparkContext: Created broadcast 255 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:50.205 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 131 (MapPartitionsRDD[465] at collect at MLTest.scala:133) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:50.205 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 131.0 with 4 tasks 20/08/03 12:50:50.206 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 131.0 (TID 449, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.206 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 131.0 (TID 450, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.206 Executor task launch worker for task 449 INFO Executor: Running task 0.0 in stage 131.0 (TID 449) 20/08/03 12:50:50.206 Executor task launch worker for task 450 INFO Executor: Running task 1.0 in stage 131.0 (TID 450) 20/08/03 12:50:50.211 Executor task launch worker for task 450 INFO Executor: Finished task 1.0 in stage 131.0 (TID 450). 1401 bytes result sent to driver 20/08/03 12:50:50.211 Executor task launch worker for task 449 INFO Executor: Finished task 0.0 in stage 131.0 (TID 449). 1401 bytes result sent to driver 20/08/03 12:50:50.211 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 131.0 (TID 451, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.211 Executor task launch worker for task 451 INFO Executor: Running task 2.0 in stage 131.0 (TID 451) 20/08/03 12:50:50.211 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 131.0 (TID 452, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.211 Executor task launch worker for task 452 INFO Executor: Running task 3.0 in stage 131.0 (TID 452) 20/08/03 12:50:50.211 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 131.0 (TID 450) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:50.211 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 131.0 (TID 449) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:50.214 Executor task launch worker for task 451 INFO Executor: Finished task 2.0 in stage 131.0 (TID 451). 1401 bytes result sent to driver 20/08/03 12:50:50.214 Executor task launch worker for task 452 INFO Executor: Finished task 3.0 in stage 131.0 (TID 452). 1401 bytes result sent to driver 20/08/03 12:50:50.214 task-result-getter-3 INFO TaskSetManager: Finished task 2.0 in stage 131.0 (TID 451) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:50.214 task-result-getter-1 INFO TaskSetManager: Finished task 3.0 in stage 131.0 (TID 452) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:50.214 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 131.0, whose tasks have all completed, from pool 20/08/03 12:50:50.214 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 131 (collect at MLTest.scala:133) finished in 0.013 s 20/08/03 12:50:50.214 dag-scheduler-event-loop INFO DAGScheduler: Job 131 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:50.215 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 131: Stage finished 20/08/03 12:50:50.215 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 131 finished: collect at MLTest.scala:133, took 0.015296 s 20/08/03 12:50:50.255 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: collect at MLTest.scala:116 20/08/03 12:50:50.255 dag-scheduler-event-loop INFO DAGScheduler: Got job 132 (collect at MLTest.scala:116) with 4 output partitions 20/08/03 12:50:50.255 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 132 (collect at MLTest.scala:116) 20/08/03 12:50:50.255 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:50.255 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:50.256 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 132 (MapPartitionsRDD[468] at collect at MLTest.scala:116), which has no missing parents 20/08/03 12:50:50.257 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_256 stored as values in memory (estimated size 15.2 KiB, free 2.1 GiB) 20/08/03 12:50:50.258 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_256_piece0 stored as bytes in memory (estimated size 5.8 KiB, free 2.1 GiB) 20/08/03 12:50:50.259 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_256_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.8 KiB, free: 2.1 GiB) 20/08/03 12:50:50.259 dag-scheduler-event-loop INFO SparkContext: Created broadcast 256 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:50.259 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 132 (MapPartitionsRDD[468] at collect at MLTest.scala:116) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:50.259 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 132.0 with 4 tasks 20/08/03 12:50:50.260 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 132.0 (TID 453, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.260 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 132.0 (TID 454, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.260 Executor task launch worker for task 454 INFO Executor: Running task 1.0 in stage 132.0 (TID 454) 20/08/03 12:50:50.260 Executor task launch worker for task 453 INFO Executor: Running task 0.0 in stage 132.0 (TID 453) 20/08/03 12:50:50.264 Executor task launch worker for task 454 INFO Executor: Finished task 1.0 in stage 132.0 (TID 454). 1970 bytes result sent to driver 20/08/03 12:50:50.264 Executor task launch worker for task 453 INFO Executor: Finished task 0.0 in stage 132.0 (TID 453). 1955 bytes result sent to driver 20/08/03 12:50:50.265 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 132.0 (TID 455, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.265 Executor task launch worker for task 455 INFO Executor: Running task 2.0 in stage 132.0 (TID 455) 20/08/03 12:50:50.265 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 132.0 (TID 456, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.265 Executor task launch worker for task 456 INFO Executor: Running task 3.0 in stage 132.0 (TID 456) 20/08/03 12:50:50.265 task-result-getter-0 INFO TaskSetManager: Finished task 1.0 in stage 132.0 (TID 454) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:50.265 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 132.0 (TID 453) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:50.268 Executor task launch worker for task 456 INFO Executor: Finished task 3.0 in stage 132.0 (TID 456). 1942 bytes result sent to driver 20/08/03 12:50:50.268 Executor task launch worker for task 455 INFO Executor: Finished task 2.0 in stage 132.0 (TID 455). 1962 bytes result sent to driver 20/08/03 12:50:50.269 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 132.0 (TID 456) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:50.269 task-result-getter-1 INFO TaskSetManager: Finished task 2.0 in stage 132.0 (TID 455) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:50.269 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 132.0, whose tasks have all completed, from pool 20/08/03 12:50:50.269 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 132 (collect at MLTest.scala:116) finished in 0.013 s 20/08/03 12:50:50.270 dag-scheduler-event-loop INFO DAGScheduler: Job 132 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:50.270 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 132: Stage finished 20/08/03 12:50:50.270 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 132 finished: collect at MLTest.scala:116, took 0.014905 s 20/08/03 12:50:50.288 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: StartStream(ProcessingTimeTrigger(0),org.apache.spark.util.SystemClock@4604213b,Map(),null) 20/08/03 12:50:50.289 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Checkpoint root /home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1740380d-40da-49b6-b740-d57be75e9c70 resolved to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1740380d-40da-49b6-b740-d57be75e9c70. 20/08/03 12:50:50.296 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1740380d-40da-49b6-b740-d57be75e9c70/metadata using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1740380d-40da-49b6-b740-d57be75e9c70/.metadata.7cd4791b-3af4-4141-a080-ef3a86f916e4.tmp 20/08/03 12:50:50.308 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1740380d-40da-49b6-b740-d57be75e9c70/.metadata.7cd4791b-3af4-4141-a080-ef3a86f916e4.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1740380d-40da-49b6-b740-d57be75e9c70/metadata 20/08/03 12:50:50.320 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Starting [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae]. Use file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1740380d-40da-49b6-b740-d57be75e9c70 to store the query checkpoint. 20/08/03 12:50:50.323 stream execution thread for [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae] INFO MicroBatchExecution: Reading table [org.apache.spark.sql.execution.streaming.MemoryStreamTable@32190e9c] from DataSourceV2 named 'memory' [org.apache.spark.sql.execution.streaming.MemoryStreamTableProvider$@490dc5fd] 20/08/03 12:50:50.325 stream execution thread for [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae] INFO MicroBatchExecution: Starting new streaming query. 20/08/03 12:50:50.325 stream execution thread for [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae] INFO MicroBatchExecution: Stream started from {} 20/08/03 12:50:50.326 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: AddData to MemoryStream[_1#1081,_2#1082]: (0.0,[6.7885086340489185,3.4576551565453197]),(0.0,[5.056501247252454,2.574601118217243]),(0.0,[6.075652199068211,3.355341616104334]),(0.0,[5.166332778860912,2.448377481827237]),(0.0,[5.684896098313557,3.7046548957911765]),(0.0,[6.50731807165581,3.004047684427792]),(0.0,[7.010912781425465,2.7780024496013715]),(1.0,[4.8414027004203835,3.211158821420722]),(1.0,[5.436985752813093,3.296990772729685]),(2.0,[4.84644814192866,3.1968908375605634]),(0.0,[7.127365231828297,3.248100610649046]),(1.0,[6.241767847456931,3.7192174938869087]),(0.0,[5.613780653545803,3.020421477260098]),(0.0,[6.882841862374561,2.915254192504282]),(1.0,[5.699513929932456,2.247074740074896]),(0.0,[7.02192590749258,2.4627220581296783]),(1.0,[4.216237488061392,2.656699174446591]),(0.0,[3.7324966114897262,2.3462678397985552]),(1.0,[5.741978867831515,3.6187835154633294]),(0.0,[5.620150427156592,3.1692083626745053]),(0.0,[5.578107805726126,2.2863316707315384]),(0.0,[5.442698469962282,2.834758817537672]),(0.0,[6.767616345179818,3.0391562262408645]),(1.0,[4.925837281912354,3.8685534349585087]),(0.0,[6.791684731921275,2.595654759591007]),(0.0,[6.731122446118569,2.2151938610676063]),(0.0,[6.092251787382735,3.1648644467226514]),(0.0,[7.007311457155493,2.3945255550358033]),(1.0,[6.067332171369262,3.301578503282249]),(0.0,[5.404295990288988,3.291892487883425]),(0.0,[7.674839072483042,2.781183265589518]),(0.0,[4.2837121498611985,3.2254558585309234]),(1.0,[6.112789784401108,2.664233955695323]),(1.0,[5.840780678327939,2.8504603244268845]),(1.0,[5.389268793409797,3.0486766295260552]),(0.0,[7.206579880655595,2.5745696091376855]),(1.0,[6.334680742393493,2.989165223067271]),(0.0,[6.39220828708044,3.5916060870142212]),(1.0,[6.574935571365726,3.1536220523404945]),(0.0,[6.604522213141694,2.734798375932801]),(0.0,[6.508320460023237,2.946338492652808]),(1.0,[5.210534043458435,2.24447246587153]),(1.0,[4.301820482847355,2.8057320879830776]),(0.0,[5.504349176848336,3.165273107673573]),(0.0,[5.444799426865632,3.251448135769693]),(0.0,[5.473220085163719,3.076888207026409]),(0.0,[6.426387726380042,2.938545841808128]),(0.0,[5.909860769315067,3.6056829798513315]),(0.0,[6.475193522926937,3.807440177850298]),(0.0,[5.963848054893707,2.623550555268978]),(1.0,[6.009222629896281,3.4475575899620026]),(0.0,[7.094718114371827,2.4454009956067817]),(1.0,[4.264010706382008,3.65338684298196]),(0.0,[5.379585321764099,3.381481745785779]),(0.0,[7.146010317633724,3.5485733241937085]),(0.0,[5.683954590364761,2.0745476381163566]),(0.0,[6.2072015922395885,3.3432172748930773]),(0.0,[5.226174068792373,3.579701566179305]),(0.0,[7.316647506340967,3.0863730559231892]),(1.0,[6.915325522847731,2.803084035949653]),(0.0,[6.479630329902845,3.239007840935509]),(0.0,[5.454587096334029,3.2374754911843655]),(2.0,[6.391989224379249,3.434901891800874]),(1.0,[6.0886711934815265,3.1971117650588363]),(0.0,[7.684738047156347,3.421241896031828]),(0.0,[4.85493416785451,3.1087723328558345]),(0.0,[6.305063368354057,2.2060572623972567]),(0.0,[7.081748274709549,2.724685616028466]),(0.0,[6.584215742308165,2.9773965559705675]),(1.0,[6.612853131358183,2.8663569212164877]),(0.0,[5.756132728856622,2.7905503944752907]),(1.0,[6.135092550527577,2.5582480777704872]),(1.0,[5.301620953306969,4.223612451290534]),(1.0,[5.9682958432514,3.8618929524452805]),(2.0,[6.207939723169083,3.439309516984485]),(0.0,[6.627109164472448,2.7520560194737964]),(0.0,[3.9259860466490686,3.0983115739148284]),(0.0,[6.0020085765576665,2.6176816137758694]),(0.0,[4.062185373981144,3.76166658403873]),(1.0,[6.2962118879304585,3.764267301123263]),(1.0,[6.14908975112769,3.590411745294196]),(1.0,[6.567608573964195,2.9306232708439306]),(0.0,[6.201805185587324,3.4958144403354416]),(1.0,[7.319347971525034,3.648290394729337]),(1.0,[6.212234548496149,3.6633867263201436]),(0.0,[5.870287402238878,2.7075846255583254]),(0.0,[5.8287726136817,2.483308333263797]),(0.0,[5.819965967504458,3.9163413775672242]),(0.0,[6.982232083397012,2.503264912876912]),(0.0,[6.084806791595145,3.72518246843444]),(0.0,[5.92962672345617,2.484757315931346]),(1.0,[7.3168542953228615,3.229242214881765]),(0.0,[5.428611939474008,2.1676964414169606]),(1.0,[6.74298764459546,2.6034237914698872]),(0.0,[7.467271990105523,3.4604682039447336]),(0.0,[6.970465097926772,4.3162388355161205]),(0.0,[5.436435936556991,2.747513090695181]),(0.0,[7.143361618211275,2.6898992927088976]),(0.0,[5.568158116555633,2.316684365492898]),(0.0,[5.40320969646403,1.929404297480336]) 20/08/03 12:50:50.326 stream execution thread for [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae] INFO MicroBatchExecution: Streaming query made progress: { "id" : "aee65039-00a8-4f3b-9bce-ee42034ac02f", "runId" : "06234527-5b5a-47f8-a9a5-29a027e7c4ae", "name" : null, "timestamp" : "2020-08-03T19:50:50.325Z", "batchId" : 0, "numInputRows" : 0, "processedRowsPerSecond" : 0.0, "durationMs" : { "latestOffset" : 0, "triggerExecution" : 0 }, "stateOperators" : [ ], "sources" : [ { "description" : "MemoryStream[_1#1081,_2#1082]", "startOffset" : null, "endOffset" : null, "numInputRows" : 0, "processedRowsPerSecond" : 0.0 } ], "sink" : { "description" : "MemorySink", "numOutputRows" : -1 } } 20/08/03 12:50:50.330 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: CheckAnswerByFunc 20/08/03 12:50:50.341 stream execution thread for [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae] INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1740380d-40da-49b6-b740-d57be75e9c70/offsets/0 using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1740380d-40da-49b6-b740-d57be75e9c70/offsets/.0.17f1e0df-2e2f-4afc-84c7-979f550788b6.tmp 20/08/03 12:50:50.348 stream execution thread for [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae] INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1740380d-40da-49b6-b740-d57be75e9c70/offsets/.0.17f1e0df-2e2f-4afc-84c7-979f550788b6.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1740380d-40da-49b6-b740-d57be75e9c70/offsets/0 20/08/03 12:50:50.348 stream execution thread for [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae] INFO MicroBatchExecution: Committed offsets for batch 0. Metadata OffsetSeqMetadata(0,1596484250336,Map(spark.sql.streaming.stateStore.providerClass -> org.apache.spark.sql.execution.streaming.state.HDFSBackedStateStoreProvider, spark.sql.streaming.join.stateFormatVersion -> 2, spark.sql.streaming.flatMapGroupsWithState.stateFormatVersion -> 2, spark.sql.streaming.multipleWatermarkPolicy -> min, spark.sql.streaming.aggregation.stateFormatVersion -> 2, spark.sql.shuffle.partitions -> 5)) 20/08/03 12:50:50.391 stream execution thread for [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae] INFO WriteToDataSourceV2Exec: Start processing data source write support: org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@7f4605a. The input RDD has 1 partitions. 20/08/03 12:50:50.391 stream execution thread for [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae] INFO SparkContext: Starting job: testStream$ at LogisticRegressionSuite.scala:39 20/08/03 12:50:50.392 dag-scheduler-event-loop INFO DAGScheduler: Got job 133 (testStream$ at LogisticRegressionSuite.scala:39) with 1 output partitions 20/08/03 12:50:50.393 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 133 (testStream$ at LogisticRegressionSuite.scala:39) 20/08/03 12:50:50.393 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:50.393 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:50.393 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 133 (MapPartitionsRDD[471] at testStream$ at LogisticRegressionSuite.scala:39), which has no missing parents 20/08/03 12:50:50.394 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_257 stored as values in memory (estimated size 22.5 KiB, free 2.1 GiB) 20/08/03 12:50:50.396 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_257_piece0 stored as bytes in memory (estimated size 9.9 KiB, free 2.1 GiB) 20/08/03 12:50:50.396 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_257_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 9.9 KiB, free: 2.1 GiB) 20/08/03 12:50:50.397 dag-scheduler-event-loop INFO SparkContext: Created broadcast 257 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:50.397 dag-scheduler-event-loop INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 133 (MapPartitionsRDD[471] at testStream$ at LogisticRegressionSuite.scala:39) (first 15 tasks are for partitions Vector(0)) 20/08/03 12:50:50.397 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 133.0 with 1 tasks 20/08/03 12:50:50.398 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 133.0 (TID 457, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 18876 bytes) 20/08/03 12:50:50.398 Executor task launch worker for task 457 INFO Executor: Running task 0.0 in stage 133.0 (TID 457) 20/08/03 12:50:50.409 Executor task launch worker for task 457 INFO DataWritingSparkTask: Commit authorized for partition 0 (task 457, attempt 0, stage 133.0) 20/08/03 12:50:50.409 Executor task launch worker for task 457 INFO DataWritingSparkTask: Committed partition 0 (task 457, attempt 0, stage 133.0) 20/08/03 12:50:50.410 Executor task launch worker for task 457 INFO Executor: Finished task 0.0 in stage 133.0 (TID 457). 6472 bytes result sent to driver 20/08/03 12:50:50.411 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 133.0 (TID 457) in 14 ms on amp-jenkins-worker-06.amp (executor driver) (1/1) 20/08/03 12:50:50.411 task-result-getter-0 INFO TaskSchedulerImpl: Removed TaskSet 133.0, whose tasks have all completed, from pool 20/08/03 12:50:50.411 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 133 (testStream$ at LogisticRegressionSuite.scala:39) finished in 0.018 s 20/08/03 12:50:50.411 dag-scheduler-event-loop INFO DAGScheduler: Job 133 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:50.411 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 133: Stage finished 20/08/03 12:50:50.411 stream execution thread for [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae] INFO DAGScheduler: Job 133 finished: testStream$ at LogisticRegressionSuite.scala:39, took 0.019996 s 20/08/03 12:50:50.412 stream execution thread for [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae] INFO WriteToDataSourceV2Exec: Data source write support org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@7f4605a is committing. 20/08/03 12:50:50.412 stream execution thread for [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae] INFO WriteToDataSourceV2Exec: Data source write support org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@7f4605a committed. 20/08/03 12:50:50.418 stream execution thread for [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae] INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1740380d-40da-49b6-b740-d57be75e9c70/commits/0 using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1740380d-40da-49b6-b740-d57be75e9c70/commits/.0.d5f2cb89-9a04-4788-93b2-fc0c64575b4f.tmp 20/08/03 12:50:50.425 stream execution thread for [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae] INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1740380d-40da-49b6-b740-d57be75e9c70/commits/.0.d5f2cb89-9a04-4788-93b2-fc0c64575b4f.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-1740380d-40da-49b6-b740-d57be75e9c70/commits/0 20/08/03 12:50:50.427 stream execution thread for [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae] INFO MicroBatchExecution: Streaming query made progress: { "id" : "aee65039-00a8-4f3b-9bce-ee42034ac02f", "runId" : "06234527-5b5a-47f8-a9a5-29a027e7c4ae", "name" : null, "timestamp" : "2020-08-03T19:50:50.336Z", "batchId" : 0, "numInputRows" : 100, "inputRowsPerSecond" : 9090.909090909092, "processedRowsPerSecond" : 1111.111111111111, "durationMs" : { "addBatch" : 36, "getBatch" : 0, "latestOffset" : 0, "queryPlanning" : 27, "triggerExecution" : 89, "walCommit" : 13 }, "stateOperators" : [ ], "sources" : [ { "description" : "MemoryStream[_1#1081,_2#1082]", "startOffset" : null, "endOffset" : 0, "numInputRows" : 100, "inputRowsPerSecond" : 9090.909090909092, "processedRowsPerSecond" : 1111.111111111111 } ], "sink" : { "description" : "MemorySink", "numOutputRows" : 100 } } 20/08/03 12:50:50.440 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Asked to cancel job group 06234527-5b5a-47f8-a9a5-29a027e7c4ae 20/08/03 12:50:50.442 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Asked to cancel job group 06234527-5b5a-47f8-a9a5-29a027e7c4ae 20/08/03 12:50:50.442 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Query [id = aee65039-00a8-4f3b-9bce-ee42034ac02f, runId = 06234527-5b5a-47f8-a9a5-29a027e7c4ae] was stopped 20/08/03 12:50:50.500 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: collect at MLTest.scala:133 20/08/03 12:50:50.500 dag-scheduler-event-loop INFO DAGScheduler: Got job 134 (collect at MLTest.scala:133) with 4 output partitions 20/08/03 12:50:50.501 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 134 (collect at MLTest.scala:133) 20/08/03 12:50:50.501 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:50.501 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:50.501 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 134 (MapPartitionsRDD[474] at collect at MLTest.scala:133), which has no missing parents 20/08/03 12:50:50.502 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_258 stored as values in memory (estimated size 26.0 KiB, free 2.1 GiB) 20/08/03 12:50:50.504 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_258_piece0 stored as bytes in memory (estimated size 10.9 KiB, free 2.1 GiB) 20/08/03 12:50:50.504 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_258_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.9 KiB, free: 2.1 GiB) 20/08/03 12:50:50.504 dag-scheduler-event-loop INFO SparkContext: Created broadcast 258 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:50.504 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 134 (MapPartitionsRDD[474] at collect at MLTest.scala:133) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:50.504 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 134.0 with 4 tasks 20/08/03 12:50:50.505 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 134.0 (TID 458, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.505 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 1.0 in stage 134.0 (TID 459, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.505 Executor task launch worker for task 459 INFO Executor: Running task 1.0 in stage 134.0 (TID 459) 20/08/03 12:50:50.505 Executor task launch worker for task 458 INFO Executor: Running task 0.0 in stage 134.0 (TID 458) 20/08/03 12:50:50.509 Executor task launch worker for task 459 INFO Executor: Finished task 1.0 in stage 134.0 (TID 459). 1403 bytes result sent to driver 20/08/03 12:50:50.509 Executor task launch worker for task 458 INFO Executor: Finished task 0.0 in stage 134.0 (TID 458). 1403 bytes result sent to driver 20/08/03 12:50:50.510 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 134.0 (TID 460, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.510 Executor task launch worker for task 460 INFO Executor: Running task 2.0 in stage 134.0 (TID 460) 20/08/03 12:50:50.510 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 134.0 (TID 461, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.510 Executor task launch worker for task 461 INFO Executor: Running task 3.0 in stage 134.0 (TID 461) 20/08/03 12:50:50.510 task-result-getter-2 INFO TaskSetManager: Finished task 1.0 in stage 134.0 (TID 459) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:50.510 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 134.0 (TID 458) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:50.513 Executor task launch worker for task 460 INFO Executor: Finished task 2.0 in stage 134.0 (TID 460). 1360 bytes result sent to driver 20/08/03 12:50:50.513 Executor task launch worker for task 461 INFO Executor: Finished task 3.0 in stage 134.0 (TID 461). 1360 bytes result sent to driver 20/08/03 12:50:50.513 task-result-getter-1 INFO TaskSetManager: Finished task 2.0 in stage 134.0 (TID 460) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:50.513 task-result-getter-0 INFO TaskSetManager: Finished task 3.0 in stage 134.0 (TID 461) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:50.513 task-result-getter-0 INFO TaskSchedulerImpl: Removed TaskSet 134.0, whose tasks have all completed, from pool 20/08/03 12:50:50.513 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 134 (collect at MLTest.scala:133) finished in 0.012 s 20/08/03 12:50:50.514 dag-scheduler-event-loop INFO DAGScheduler: Job 134 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:50.514 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 134: Stage finished 20/08/03 12:50:50.514 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 134 finished: collect at MLTest.scala:133, took 0.014074 s 20/08/03 12:50:50.554 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: collect at MLTest.scala:116 20/08/03 12:50:50.554 dag-scheduler-event-loop INFO DAGScheduler: Got job 135 (collect at MLTest.scala:116) with 4 output partitions 20/08/03 12:50:50.554 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 135 (collect at MLTest.scala:116) 20/08/03 12:50:50.554 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:50.554 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:50.555 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 135 (MapPartitionsRDD[477] at collect at MLTest.scala:116), which has no missing parents 20/08/03 12:50:50.555 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_259 stored as values in memory (estimated size 15.2 KiB, free 2.1 GiB) 20/08/03 12:50:50.557 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_259_piece0 stored as bytes in memory (estimated size 5.8 KiB, free 2.1 GiB) 20/08/03 12:50:50.557 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_259_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.8 KiB, free: 2.1 GiB) 20/08/03 12:50:50.557 dag-scheduler-event-loop INFO SparkContext: Created broadcast 259 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:50.557 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 135 (MapPartitionsRDD[477] at collect at MLTest.scala:116) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:50.557 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 135.0 with 4 tasks 20/08/03 12:50:50.558 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 135.0 (TID 462, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.558 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 135.0 (TID 463, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.558 Executor task launch worker for task 462 INFO Executor: Running task 0.0 in stage 135.0 (TID 462) 20/08/03 12:50:50.558 Executor task launch worker for task 463 INFO Executor: Running task 1.0 in stage 135.0 (TID 463) 20/08/03 12:50:50.562 Executor task launch worker for task 462 INFO Executor: Finished task 0.0 in stage 135.0 (TID 462). 1955 bytes result sent to driver 20/08/03 12:50:50.562 Executor task launch worker for task 463 INFO Executor: Finished task 1.0 in stage 135.0 (TID 463). 1970 bytes result sent to driver 20/08/03 12:50:50.562 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 135.0 (TID 464, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.562 Executor task launch worker for task 464 INFO Executor: Running task 2.0 in stage 135.0 (TID 464) 20/08/03 12:50:50.562 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 135.0 (TID 465, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.562 Executor task launch worker for task 465 INFO Executor: Running task 3.0 in stage 135.0 (TID 465) 20/08/03 12:50:50.562 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 135.0 (TID 462) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:50.562 task-result-getter-3 INFO TaskSetManager: Finished task 1.0 in stage 135.0 (TID 463) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:50.563 Executor task launch worker for task 464 INFO Executor: Finished task 2.0 in stage 135.0 (TID 464). 1876 bytes result sent to driver 20/08/03 12:50:50.564 task-result-getter-1 INFO TaskSetManager: Finished task 2.0 in stage 135.0 (TID 464) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:50.564 Executor task launch worker for task 465 INFO Executor: Finished task 3.0 in stage 135.0 (TID 465). 1856 bytes result sent to driver 20/08/03 12:50:50.564 task-result-getter-0 INFO TaskSetManager: Finished task 3.0 in stage 135.0 (TID 465) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:50.564 task-result-getter-0 INFO TaskSchedulerImpl: Removed TaskSet 135.0, whose tasks have all completed, from pool 20/08/03 12:50:50.564 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 135 (collect at MLTest.scala:116) finished in 0.009 s 20/08/03 12:50:50.565 dag-scheduler-event-loop INFO DAGScheduler: Job 135 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:50.565 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 135: Stage finished 20/08/03 12:50:50.565 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 135 finished: collect at MLTest.scala:116, took 0.010754 s 20/08/03 12:50:50.584 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: StartStream(ProcessingTimeTrigger(0),org.apache.spark.util.SystemClock@2514dc89,Map(),null) 20/08/03 12:50:50.586 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Checkpoint root /home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-cf5c0e47-9a31-46c6-a5d3-cdad9bf37a48 resolved to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-cf5c0e47-9a31-46c6-a5d3-cdad9bf37a48. 20/08/03 12:50:50.591 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-cf5c0e47-9a31-46c6-a5d3-cdad9bf37a48/metadata using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-cf5c0e47-9a31-46c6-a5d3-cdad9bf37a48/.metadata.8e69af28-2116-4da9-94a7-1a984589cb7a.tmp 20/08/03 12:50:50.600 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-cf5c0e47-9a31-46c6-a5d3-cdad9bf37a48/.metadata.8e69af28-2116-4da9-94a7-1a984589cb7a.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-cf5c0e47-9a31-46c6-a5d3-cdad9bf37a48/metadata 20/08/03 12:50:50.607 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Starting [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3]. Use file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-cf5c0e47-9a31-46c6-a5d3-cdad9bf37a48 to store the query checkpoint. 20/08/03 12:50:50.610 stream execution thread for [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3] INFO MicroBatchExecution: Reading table [org.apache.spark.sql.execution.streaming.MemoryStreamTable@57a2dec3] from DataSourceV2 named 'memory' [org.apache.spark.sql.execution.streaming.MemoryStreamTableProvider$@490dc5fd] 20/08/03 12:50:50.611 stream execution thread for [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3] INFO MicroBatchExecution: Starting new streaming query. 20/08/03 12:50:50.611 stream execution thread for [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3] INFO MicroBatchExecution: Stream started from {} 20/08/03 12:50:50.611 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: AddData to MemoryStream[_1#1161,_2#1162]: (0.0,[6.7885086340489185,3.4576551565453197]),(0.0,[5.056501247252454,2.574601118217243]),(0.0,[6.075652199068211,3.355341616104334]),(0.0,[5.166332778860912,2.448377481827237]),(0.0,[5.684896098313557,3.7046548957911765]),(0.0,[6.50731807165581,3.004047684427792]),(0.0,[7.010912781425465,2.7780024496013715]),(1.0,[4.8414027004203835,3.211158821420722]),(1.0,[5.436985752813093,3.296990772729685]),(2.0,[4.84644814192866,3.1968908375605634]),(0.0,[7.127365231828297,3.248100610649046]),(1.0,[6.241767847456931,3.7192174938869087]),(0.0,[5.613780653545803,3.020421477260098]),(0.0,[6.882841862374561,2.915254192504282]),(1.0,[5.699513929932456,2.247074740074896]),(0.0,[7.02192590749258,2.4627220581296783]),(1.0,[4.216237488061392,2.656699174446591]),(0.0,[3.7324966114897262,2.3462678397985552]),(1.0,[5.741978867831515,3.6187835154633294]),(0.0,[5.620150427156592,3.1692083626745053]),(0.0,[5.578107805726126,2.2863316707315384]),(0.0,[5.442698469962282,2.834758817537672]),(0.0,[6.767616345179818,3.0391562262408645]),(1.0,[4.925837281912354,3.8685534349585087]),(0.0,[6.791684731921275,2.595654759591007]),(0.0,[6.731122446118569,2.2151938610676063]),(0.0,[6.092251787382735,3.1648644467226514]),(0.0,[7.007311457155493,2.3945255550358033]),(1.0,[6.067332171369262,3.301578503282249]),(0.0,[5.404295990288988,3.291892487883425]),(0.0,[7.674839072483042,2.781183265589518]),(0.0,[4.2837121498611985,3.2254558585309234]),(1.0,[6.112789784401108,2.664233955695323]),(1.0,[5.840780678327939,2.8504603244268845]),(1.0,[5.389268793409797,3.0486766295260552]),(0.0,[7.206579880655595,2.5745696091376855]),(1.0,[6.334680742393493,2.989165223067271]),(0.0,[6.39220828708044,3.5916060870142212]),(1.0,[6.574935571365726,3.1536220523404945]),(0.0,[6.604522213141694,2.734798375932801]),(0.0,[6.508320460023237,2.946338492652808]),(1.0,[5.210534043458435,2.24447246587153]),(1.0,[4.301820482847355,2.8057320879830776]),(0.0,[5.504349176848336,3.165273107673573]),(0.0,[5.444799426865632,3.251448135769693]),(0.0,[5.473220085163719,3.076888207026409]),(0.0,[6.426387726380042,2.938545841808128]),(0.0,[5.909860769315067,3.6056829798513315]),(0.0,[6.475193522926937,3.807440177850298]),(0.0,[5.963848054893707,2.623550555268978]),(1.0,[6.009222629896281,3.4475575899620026]),(0.0,[7.094718114371827,2.4454009956067817]),(1.0,[4.264010706382008,3.65338684298196]),(0.0,[5.379585321764099,3.381481745785779]),(0.0,[7.146010317633724,3.5485733241937085]),(0.0,[5.683954590364761,2.0745476381163566]),(0.0,[6.2072015922395885,3.3432172748930773]),(0.0,[5.226174068792373,3.579701566179305]),(0.0,[7.316647506340967,3.0863730559231892]),(1.0,[6.915325522847731,2.803084035949653]),(0.0,[6.479630329902845,3.239007840935509]),(0.0,[5.454587096334029,3.2374754911843655]),(2.0,[6.391989224379249,3.434901891800874]),(1.0,[6.0886711934815265,3.1971117650588363]),(0.0,[7.684738047156347,3.421241896031828]),(0.0,[4.85493416785451,3.1087723328558345]),(0.0,[6.305063368354057,2.2060572623972567]),(0.0,[7.081748274709549,2.724685616028466]),(0.0,[6.584215742308165,2.9773965559705675]),(1.0,[6.612853131358183,2.8663569212164877]),(0.0,[5.756132728856622,2.7905503944752907]),(1.0,[6.135092550527577,2.5582480777704872]),(1.0,[5.301620953306969,4.223612451290534]),(1.0,[5.9682958432514,3.8618929524452805]),(2.0,[6.207939723169083,3.439309516984485]),(0.0,[6.627109164472448,2.7520560194737964]),(0.0,[3.9259860466490686,3.0983115739148284]),(0.0,[6.0020085765576665,2.6176816137758694]),(0.0,[4.062185373981144,3.76166658403873]),(1.0,[6.2962118879304585,3.764267301123263]),(1.0,[6.14908975112769,3.590411745294196]),(1.0,[6.567608573964195,2.9306232708439306]),(0.0,[6.201805185587324,3.4958144403354416]),(1.0,[7.319347971525034,3.648290394729337]),(1.0,[6.212234548496149,3.6633867263201436]),(0.0,[5.870287402238878,2.7075846255583254]),(0.0,[5.8287726136817,2.483308333263797]),(0.0,[5.819965967504458,3.9163413775672242]),(0.0,[6.982232083397012,2.503264912876912]),(0.0,[6.084806791595145,3.72518246843444]),(0.0,[5.92962672345617,2.484757315931346]),(1.0,[7.3168542953228615,3.229242214881765]),(0.0,[5.428611939474008,2.1676964414169606]),(1.0,[6.74298764459546,2.6034237914698872]),(0.0,[7.467271990105523,3.4604682039447336]),(0.0,[6.970465097926772,4.3162388355161205]),(0.0,[5.436435936556991,2.747513090695181]),(0.0,[7.143361618211275,2.6898992927088976]),(0.0,[5.568158116555633,2.316684365492898]),(0.0,[5.40320969646403,1.929404297480336]) 20/08/03 12:50:50.612 stream execution thread for [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3] INFO MicroBatchExecution: Streaming query made progress: { "id" : "ccb0d54f-ecb0-4380-9b90-b401ddf92525", "runId" : "f6cd630e-8cef-4fc6-83e0-451f657a46f3", "name" : null, "timestamp" : "2020-08-03T19:50:50.611Z", "batchId" : 0, "numInputRows" : 0, "processedRowsPerSecond" : 0.0, "durationMs" : { "latestOffset" : 0, "triggerExecution" : 1 }, "stateOperators" : [ ], "sources" : [ { "description" : "MemoryStream[_1#1161,_2#1162]", "startOffset" : null, "endOffset" : null, "numInputRows" : 0, "processedRowsPerSecond" : 0.0 } ], "sink" : { "description" : "MemorySink", "numOutputRows" : -1 } } 20/08/03 12:50:50.614 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: CheckAnswerByFunc 20/08/03 12:50:50.628 stream execution thread for [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3] INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-cf5c0e47-9a31-46c6-a5d3-cdad9bf37a48/offsets/0 using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-cf5c0e47-9a31-46c6-a5d3-cdad9bf37a48/offsets/.0.99a371cb-8174-459d-b9e8-55d3148ed0b6.tmp 20/08/03 12:50:50.637 stream execution thread for [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3] INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-cf5c0e47-9a31-46c6-a5d3-cdad9bf37a48/offsets/.0.99a371cb-8174-459d-b9e8-55d3148ed0b6.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-cf5c0e47-9a31-46c6-a5d3-cdad9bf37a48/offsets/0 20/08/03 12:50:50.637 stream execution thread for [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3] INFO MicroBatchExecution: Committed offsets for batch 0. Metadata OffsetSeqMetadata(0,1596484250623,Map(spark.sql.streaming.stateStore.providerClass -> org.apache.spark.sql.execution.streaming.state.HDFSBackedStateStoreProvider, spark.sql.streaming.join.stateFormatVersion -> 2, spark.sql.streaming.flatMapGroupsWithState.stateFormatVersion -> 2, spark.sql.streaming.multipleWatermarkPolicy -> min, spark.sql.streaming.aggregation.stateFormatVersion -> 2, spark.sql.shuffle.partitions -> 5)) 20/08/03 12:50:50.681 stream execution thread for [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3] INFO WriteToDataSourceV2Exec: Start processing data source write support: org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@1858307c. The input RDD has 1 partitions. 20/08/03 12:50:50.681 stream execution thread for [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3] INFO SparkContext: Starting job: testStream$ at LogisticRegressionSuite.scala:39 20/08/03 12:50:50.682 dag-scheduler-event-loop INFO DAGScheduler: Got job 136 (testStream$ at LogisticRegressionSuite.scala:39) with 1 output partitions 20/08/03 12:50:50.682 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 136 (testStream$ at LogisticRegressionSuite.scala:39) 20/08/03 12:50:50.682 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:50.682 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:50.682 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 136 (MapPartitionsRDD[480] at testStream$ at LogisticRegressionSuite.scala:39), which has no missing parents 20/08/03 12:50:50.684 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_260 stored as values in memory (estimated size 22.5 KiB, free 2.1 GiB) 20/08/03 12:50:50.685 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_260_piece0 stored as bytes in memory (estimated size 9.9 KiB, free 2.1 GiB) 20/08/03 12:50:50.685 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_260_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 9.9 KiB, free: 2.1 GiB) 20/08/03 12:50:50.685 dag-scheduler-event-loop INFO SparkContext: Created broadcast 260 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:50.686 dag-scheduler-event-loop INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 136 (MapPartitionsRDD[480] at testStream$ at LogisticRegressionSuite.scala:39) (first 15 tasks are for partitions Vector(0)) 20/08/03 12:50:50.686 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 136.0 with 1 tasks 20/08/03 12:50:50.686 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 136.0 (TID 466, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 18876 bytes) 20/08/03 12:50:50.686 Executor task launch worker for task 466 INFO Executor: Running task 0.0 in stage 136.0 (TID 466) 20/08/03 12:50:50.695 Executor task launch worker for task 466 INFO DataWritingSparkTask: Commit authorized for partition 0 (task 466, attempt 0, stage 136.0) 20/08/03 12:50:50.695 Executor task launch worker for task 466 INFO DataWritingSparkTask: Committed partition 0 (task 466, attempt 0, stage 136.0) 20/08/03 12:50:50.696 Executor task launch worker for task 466 INFO Executor: Finished task 0.0 in stage 136.0 (TID 466). 6472 bytes result sent to driver 20/08/03 12:50:50.696 task-result-getter-2 INFO TaskSetManager: Finished task 0.0 in stage 136.0 (TID 466) in 10 ms on amp-jenkins-worker-06.amp (executor driver) (1/1) 20/08/03 12:50:50.696 task-result-getter-2 INFO TaskSchedulerImpl: Removed TaskSet 136.0, whose tasks have all completed, from pool 20/08/03 12:50:50.697 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 136 (testStream$ at LogisticRegressionSuite.scala:39) finished in 0.014 s 20/08/03 12:50:50.697 dag-scheduler-event-loop INFO DAGScheduler: Job 136 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:50.697 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 136: Stage finished 20/08/03 12:50:50.697 stream execution thread for [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3] INFO DAGScheduler: Job 136 finished: testStream$ at LogisticRegressionSuite.scala:39, took 0.015697 s 20/08/03 12:50:50.697 stream execution thread for [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3] INFO WriteToDataSourceV2Exec: Data source write support org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@1858307c is committing. 20/08/03 12:50:50.697 stream execution thread for [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3] INFO WriteToDataSourceV2Exec: Data source write support org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@1858307c committed. 20/08/03 12:50:50.702 stream execution thread for [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3] INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-cf5c0e47-9a31-46c6-a5d3-cdad9bf37a48/commits/0 using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-cf5c0e47-9a31-46c6-a5d3-cdad9bf37a48/commits/.0.1eeecb81-043d-4a16-ab9a-1aba13052354.tmp 20/08/03 12:50:50.709 stream execution thread for [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3] INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-cf5c0e47-9a31-46c6-a5d3-cdad9bf37a48/commits/.0.1eeecb81-043d-4a16-ab9a-1aba13052354.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-cf5c0e47-9a31-46c6-a5d3-cdad9bf37a48/commits/0 20/08/03 12:50:50.710 stream execution thread for [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3] INFO MicroBatchExecution: Streaming query made progress: { "id" : "ccb0d54f-ecb0-4380-9b90-b401ddf92525", "runId" : "f6cd630e-8cef-4fc6-83e0-451f657a46f3", "name" : null, "timestamp" : "2020-08-03T19:50:50.623Z", "batchId" : 0, "numInputRows" : 100, "inputRowsPerSecond" : 8333.333333333334, "processedRowsPerSecond" : 1162.7906976744187, "durationMs" : { "addBatch" : 29, "getBatch" : 0, "latestOffset" : 0, "queryPlanning" : 30, "triggerExecution" : 86, "walCommit" : 14 }, "stateOperators" : [ ], "sources" : [ { "description" : "MemoryStream[_1#1161,_2#1162]", "startOffset" : null, "endOffset" : 0, "numInputRows" : 100, "inputRowsPerSecond" : 8333.333333333334, "processedRowsPerSecond" : 1162.7906976744187 } ], "sink" : { "description" : "MemorySink", "numOutputRows" : 100 } } 20/08/03 12:50:50.712 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Asked to cancel job group f6cd630e-8cef-4fc6-83e0-451f657a46f3 20/08/03 12:50:50.713 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Asked to cancel job group f6cd630e-8cef-4fc6-83e0-451f657a46f3 20/08/03 12:50:50.713 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Query [id = ccb0d54f-ecb0-4380-9b90-b401ddf92525, runId = f6cd630e-8cef-4fc6-83e0-451f657a46f3] was stopped 20/08/03 12:50:50.762 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: collect at MLTest.scala:133 20/08/03 12:50:50.763 dag-scheduler-event-loop INFO DAGScheduler: Got job 137 (collect at MLTest.scala:133) with 4 output partitions 20/08/03 12:50:50.764 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 137 (collect at MLTest.scala:133) 20/08/03 12:50:50.764 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:50.764 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:50.764 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 137 (MapPartitionsRDD[483] at collect at MLTest.scala:133), which has no missing parents 20/08/03 12:50:50.765 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_261 stored as values in memory (estimated size 26.0 KiB, free 2.1 GiB) 20/08/03 12:50:50.767 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_261_piece0 stored as bytes in memory (estimated size 10.9 KiB, free 2.1 GiB) 20/08/03 12:50:50.767 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_261_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.9 KiB, free: 2.1 GiB) 20/08/03 12:50:50.767 dag-scheduler-event-loop INFO SparkContext: Created broadcast 261 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:50.767 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 137 (MapPartitionsRDD[483] at collect at MLTest.scala:133) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:50.767 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 137.0 with 4 tasks 20/08/03 12:50:50.768 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 137.0 (TID 467, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.768 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 137.0 (TID 468, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.768 Executor task launch worker for task 467 INFO Executor: Running task 0.0 in stage 137.0 (TID 467) 20/08/03 12:50:50.768 Executor task launch worker for task 468 INFO Executor: Running task 1.0 in stage 137.0 (TID 468) 20/08/03 12:50:50.772 Executor task launch worker for task 467 INFO Executor: Finished task 0.0 in stage 137.0 (TID 467). 1402 bytes result sent to driver 20/08/03 12:50:50.772 Executor task launch worker for task 468 INFO Executor: Finished task 1.0 in stage 137.0 (TID 468). 1402 bytes result sent to driver 20/08/03 12:50:50.772 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 137.0 (TID 469, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.772 Executor task launch worker for task 469 INFO Executor: Running task 2.0 in stage 137.0 (TID 469) 20/08/03 12:50:50.772 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 137.0 (TID 470, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.772 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 137.0 (TID 467) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:50.773 Executor task launch worker for task 470 INFO Executor: Running task 3.0 in stage 137.0 (TID 470) 20/08/03 12:50:50.773 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 137.0 (TID 468) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:50.776 Executor task launch worker for task 469 INFO Executor: Finished task 2.0 in stage 137.0 (TID 469). 1359 bytes result sent to driver 20/08/03 12:50:50.776 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 137.0 (TID 469) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:50.776 Executor task launch worker for task 470 INFO Executor: Finished task 3.0 in stage 137.0 (TID 470). 1402 bytes result sent to driver 20/08/03 12:50:50.776 task-result-getter-2 INFO TaskSetManager: Finished task 3.0 in stage 137.0 (TID 470) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:50.776 task-result-getter-2 INFO TaskSchedulerImpl: Removed TaskSet 137.0, whose tasks have all completed, from pool 20/08/03 12:50:50.776 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 137 (collect at MLTest.scala:133) finished in 0.012 s 20/08/03 12:50:50.777 dag-scheduler-event-loop INFO DAGScheduler: Job 137 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:50.777 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 137: Stage finished 20/08/03 12:50:50.777 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 137 finished: collect at MLTest.scala:133, took 0.014549 s 20/08/03 12:50:50.817 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: collect at MLTest.scala:116 20/08/03 12:50:50.818 dag-scheduler-event-loop INFO DAGScheduler: Got job 138 (collect at MLTest.scala:116) with 4 output partitions 20/08/03 12:50:50.818 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 138 (collect at MLTest.scala:116) 20/08/03 12:50:50.818 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:50.818 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:50.818 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 138 (MapPartitionsRDD[486] at collect at MLTest.scala:116), which has no missing parents 20/08/03 12:50:50.819 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_262 stored as values in memory (estimated size 15.2 KiB, free 2.1 GiB) 20/08/03 12:50:50.821 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_262_piece0 stored as bytes in memory (estimated size 5.8 KiB, free 2.1 GiB) 20/08/03 12:50:50.821 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_262_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.8 KiB, free: 2.1 GiB) 20/08/03 12:50:50.821 dag-scheduler-event-loop INFO SparkContext: Created broadcast 262 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:50.821 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 138 (MapPartitionsRDD[486] at collect at MLTest.scala:116) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:50.821 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 138.0 with 4 tasks 20/08/03 12:50:50.822 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 138.0 (TID 471, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.822 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 138.0 (TID 472, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.822 Executor task launch worker for task 471 INFO Executor: Running task 0.0 in stage 138.0 (TID 471) 20/08/03 12:50:50.822 Executor task launch worker for task 472 INFO Executor: Running task 1.0 in stage 138.0 (TID 472) 20/08/03 12:50:50.826 Executor task launch worker for task 471 INFO Executor: Finished task 0.0 in stage 138.0 (TID 471). 1955 bytes result sent to driver 20/08/03 12:50:50.827 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 2.0 in stage 138.0 (TID 473, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.827 Executor task launch worker for task 473 INFO Executor: Running task 2.0 in stage 138.0 (TID 473) 20/08/03 12:50:50.827 Executor task launch worker for task 472 INFO Executor: Finished task 1.0 in stage 138.0 (TID 472). 1970 bytes result sent to driver 20/08/03 12:50:50.827 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 138.0 (TID 471) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:50.827 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 3.0 in stage 138.0 (TID 474, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:50.827 Executor task launch worker for task 474 INFO Executor: Running task 3.0 in stage 138.0 (TID 474) 20/08/03 12:50:50.827 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 138.0 (TID 472) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:50.828 Executor task launch worker for task 473 INFO Executor: Finished task 2.0 in stage 138.0 (TID 473). 1876 bytes result sent to driver 20/08/03 12:50:50.828 Executor task launch worker for task 474 INFO Executor: Finished task 3.0 in stage 138.0 (TID 474). 1856 bytes result sent to driver 20/08/03 12:50:50.828 task-result-getter-0 INFO TaskSetManager: Finished task 2.0 in stage 138.0 (TID 473) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:50.829 task-result-getter-2 INFO TaskSetManager: Finished task 3.0 in stage 138.0 (TID 474) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:50.829 task-result-getter-2 INFO TaskSchedulerImpl: Removed TaskSet 138.0, whose tasks have all completed, from pool 20/08/03 12:50:50.829 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 138 (collect at MLTest.scala:116) finished in 0.010 s 20/08/03 12:50:50.829 dag-scheduler-event-loop INFO DAGScheduler: Job 138 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:50.829 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 138: Stage finished 20/08/03 12:50:50.829 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 138 finished: collect at MLTest.scala:116, took 0.011706 s 20/08/03 12:50:50.849 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: StartStream(ProcessingTimeTrigger(0),org.apache.spark.util.SystemClock@44c9239b,Map(),null) 20/08/03 12:50:50.850 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Checkpoint root /home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-bc0726a9-e9ea-4957-9091-2da4bc53a616 resolved to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-bc0726a9-e9ea-4957-9091-2da4bc53a616. 20/08/03 12:50:50.856 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-bc0726a9-e9ea-4957-9091-2da4bc53a616/metadata using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-bc0726a9-e9ea-4957-9091-2da4bc53a616/.metadata.f008f40b-d8f7-4abc-a7fc-8cc4cc90a97b.tmp 20/08/03 12:50:50.865 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-bc0726a9-e9ea-4957-9091-2da4bc53a616/.metadata.f008f40b-d8f7-4abc-a7fc-8cc4cc90a97b.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-bc0726a9-e9ea-4957-9091-2da4bc53a616/metadata 20/08/03 12:50:50.876 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Starting [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f]. Use file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-bc0726a9-e9ea-4957-9091-2da4bc53a616 to store the query checkpoint. 20/08/03 12:50:50.880 stream execution thread for [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f] INFO MicroBatchExecution: Reading table [org.apache.spark.sql.execution.streaming.MemoryStreamTable@7120b603] from DataSourceV2 named 'memory' [org.apache.spark.sql.execution.streaming.MemoryStreamTableProvider$@490dc5fd] 20/08/03 12:50:50.881 stream execution thread for [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f] INFO MicroBatchExecution: Starting new streaming query. 20/08/03 12:50:50.881 stream execution thread for [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f] INFO MicroBatchExecution: Stream started from {} 20/08/03 12:50:50.881 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: AddData to MemoryStream[_1#1241,_2#1242]: (0.0,[6.7885086340489185,3.4576551565453197]),(0.0,[5.056501247252454,2.574601118217243]),(0.0,[6.075652199068211,3.355341616104334]),(0.0,[5.166332778860912,2.448377481827237]),(0.0,[5.684896098313557,3.7046548957911765]),(0.0,[6.50731807165581,3.004047684427792]),(0.0,[7.010912781425465,2.7780024496013715]),(1.0,[4.8414027004203835,3.211158821420722]),(1.0,[5.436985752813093,3.296990772729685]),(2.0,[4.84644814192866,3.1968908375605634]),(0.0,[7.127365231828297,3.248100610649046]),(1.0,[6.241767847456931,3.7192174938869087]),(0.0,[5.613780653545803,3.020421477260098]),(0.0,[6.882841862374561,2.915254192504282]),(1.0,[5.699513929932456,2.247074740074896]),(0.0,[7.02192590749258,2.4627220581296783]),(1.0,[4.216237488061392,2.656699174446591]),(0.0,[3.7324966114897262,2.3462678397985552]),(1.0,[5.741978867831515,3.6187835154633294]),(0.0,[5.620150427156592,3.1692083626745053]),(0.0,[5.578107805726126,2.2863316707315384]),(0.0,[5.442698469962282,2.834758817537672]),(0.0,[6.767616345179818,3.0391562262408645]),(1.0,[4.925837281912354,3.8685534349585087]),(0.0,[6.791684731921275,2.595654759591007]),(0.0,[6.731122446118569,2.2151938610676063]),(0.0,[6.092251787382735,3.1648644467226514]),(0.0,[7.007311457155493,2.3945255550358033]),(1.0,[6.067332171369262,3.301578503282249]),(0.0,[5.404295990288988,3.291892487883425]),(0.0,[7.674839072483042,2.781183265589518]),(0.0,[4.2837121498611985,3.2254558585309234]),(1.0,[6.112789784401108,2.664233955695323]),(1.0,[5.840780678327939,2.8504603244268845]),(1.0,[5.389268793409797,3.0486766295260552]),(0.0,[7.206579880655595,2.5745696091376855]),(1.0,[6.334680742393493,2.989165223067271]),(0.0,[6.39220828708044,3.5916060870142212]),(1.0,[6.574935571365726,3.1536220523404945]),(0.0,[6.604522213141694,2.734798375932801]),(0.0,[6.508320460023237,2.946338492652808]),(1.0,[5.210534043458435,2.24447246587153]),(1.0,[4.301820482847355,2.8057320879830776]),(0.0,[5.504349176848336,3.165273107673573]),(0.0,[5.444799426865632,3.251448135769693]),(0.0,[5.473220085163719,3.076888207026409]),(0.0,[6.426387726380042,2.938545841808128]),(0.0,[5.909860769315067,3.6056829798513315]),(0.0,[6.475193522926937,3.807440177850298]),(0.0,[5.963848054893707,2.623550555268978]),(1.0,[6.009222629896281,3.4475575899620026]),(0.0,[7.094718114371827,2.4454009956067817]),(1.0,[4.264010706382008,3.65338684298196]),(0.0,[5.379585321764099,3.381481745785779]),(0.0,[7.146010317633724,3.5485733241937085]),(0.0,[5.683954590364761,2.0745476381163566]),(0.0,[6.2072015922395885,3.3432172748930773]),(0.0,[5.226174068792373,3.579701566179305]),(0.0,[7.316647506340967,3.0863730559231892]),(1.0,[6.915325522847731,2.803084035949653]),(0.0,[6.479630329902845,3.239007840935509]),(0.0,[5.454587096334029,3.2374754911843655]),(2.0,[6.391989224379249,3.434901891800874]),(1.0,[6.0886711934815265,3.1971117650588363]),(0.0,[7.684738047156347,3.421241896031828]),(0.0,[4.85493416785451,3.1087723328558345]),(0.0,[6.305063368354057,2.2060572623972567]),(0.0,[7.081748274709549,2.724685616028466]),(0.0,[6.584215742308165,2.9773965559705675]),(1.0,[6.612853131358183,2.8663569212164877]),(0.0,[5.756132728856622,2.7905503944752907]),(1.0,[6.135092550527577,2.5582480777704872]),(1.0,[5.301620953306969,4.223612451290534]),(1.0,[5.9682958432514,3.8618929524452805]),(2.0,[6.207939723169083,3.439309516984485]),(0.0,[6.627109164472448,2.7520560194737964]),(0.0,[3.9259860466490686,3.0983115739148284]),(0.0,[6.0020085765576665,2.6176816137758694]),(0.0,[4.062185373981144,3.76166658403873]),(1.0,[6.2962118879304585,3.764267301123263]),(1.0,[6.14908975112769,3.590411745294196]),(1.0,[6.567608573964195,2.9306232708439306]),(0.0,[6.201805185587324,3.4958144403354416]),(1.0,[7.319347971525034,3.648290394729337]),(1.0,[6.212234548496149,3.6633867263201436]),(0.0,[5.870287402238878,2.7075846255583254]),(0.0,[5.8287726136817,2.483308333263797]),(0.0,[5.819965967504458,3.9163413775672242]),(0.0,[6.982232083397012,2.503264912876912]),(0.0,[6.084806791595145,3.72518246843444]),(0.0,[5.92962672345617,2.484757315931346]),(1.0,[7.3168542953228615,3.229242214881765]),(0.0,[5.428611939474008,2.1676964414169606]),(1.0,[6.74298764459546,2.6034237914698872]),(0.0,[7.467271990105523,3.4604682039447336]),(0.0,[6.970465097926772,4.3162388355161205]),(0.0,[5.436435936556991,2.747513090695181]),(0.0,[7.143361618211275,2.6898992927088976]),(0.0,[5.568158116555633,2.316684365492898]),(0.0,[5.40320969646403,1.929404297480336]) 20/08/03 12:50:50.882 stream execution thread for [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f] INFO MicroBatchExecution: Streaming query made progress: { "id" : "48d85991-9680-4d11-a4c1-261bad7ee0ce", "runId" : "2cf71392-a13a-46c8-a8a6-2037b3a96f2f", "name" : null, "timestamp" : "2020-08-03T19:50:50.881Z", "batchId" : 0, "numInputRows" : 0, "processedRowsPerSecond" : 0.0, "durationMs" : { "latestOffset" : 0, "triggerExecution" : 1 }, "stateOperators" : [ ], "sources" : [ { "description" : "MemoryStream[_1#1241,_2#1242]", "startOffset" : null, "endOffset" : null, "numInputRows" : 0, "processedRowsPerSecond" : 0.0 } ], "sink" : { "description" : "MemorySink", "numOutputRows" : -1 } } 20/08/03 12:50:50.885 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: CheckAnswerByFunc 20/08/03 12:50:50.898 stream execution thread for [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f] INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-bc0726a9-e9ea-4957-9091-2da4bc53a616/offsets/0 using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-bc0726a9-e9ea-4957-9091-2da4bc53a616/offsets/.0.4dc6a5f4-22f1-4807-81c3-771be1fc261a.tmp 20/08/03 12:50:50.906 stream execution thread for [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f] INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-bc0726a9-e9ea-4957-9091-2da4bc53a616/offsets/.0.4dc6a5f4-22f1-4807-81c3-771be1fc261a.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-bc0726a9-e9ea-4957-9091-2da4bc53a616/offsets/0 20/08/03 12:50:50.906 stream execution thread for [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f] INFO MicroBatchExecution: Committed offsets for batch 0. Metadata OffsetSeqMetadata(0,1596484250893,Map(spark.sql.streaming.stateStore.providerClass -> org.apache.spark.sql.execution.streaming.state.HDFSBackedStateStoreProvider, spark.sql.streaming.join.stateFormatVersion -> 2, spark.sql.streaming.flatMapGroupsWithState.stateFormatVersion -> 2, spark.sql.streaming.multipleWatermarkPolicy -> min, spark.sql.streaming.aggregation.stateFormatVersion -> 2, spark.sql.shuffle.partitions -> 5)) 20/08/03 12:50:50.943 stream execution thread for [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f] INFO WriteToDataSourceV2Exec: Start processing data source write support: org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@4abe651a. The input RDD has 1 partitions. 20/08/03 12:50:50.944 stream execution thread for [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f] INFO SparkContext: Starting job: testStream$ at LogisticRegressionSuite.scala:39 20/08/03 12:50:50.944 dag-scheduler-event-loop INFO DAGScheduler: Got job 139 (testStream$ at LogisticRegressionSuite.scala:39) with 1 output partitions 20/08/03 12:50:50.945 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 139 (testStream$ at LogisticRegressionSuite.scala:39) 20/08/03 12:50:50.945 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:50.945 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:50.945 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 139 (MapPartitionsRDD[489] at testStream$ at LogisticRegressionSuite.scala:39), which has no missing parents 20/08/03 12:50:50.946 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_263 stored as values in memory (estimated size 22.5 KiB, free 2.1 GiB) 20/08/03 12:50:50.948 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_263_piece0 stored as bytes in memory (estimated size 9.9 KiB, free 2.1 GiB) 20/08/03 12:50:50.948 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_263_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 9.9 KiB, free: 2.1 GiB) 20/08/03 12:50:50.949 dag-scheduler-event-loop INFO SparkContext: Created broadcast 263 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:50.949 dag-scheduler-event-loop INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 139 (MapPartitionsRDD[489] at testStream$ at LogisticRegressionSuite.scala:39) (first 15 tasks are for partitions Vector(0)) 20/08/03 12:50:50.949 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 139.0 with 1 tasks 20/08/03 12:50:50.949 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 139.0 (TID 475, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 18876 bytes) 20/08/03 12:50:50.950 Executor task launch worker for task 475 INFO Executor: Running task 0.0 in stage 139.0 (TID 475) 20/08/03 12:50:50.956 Executor task launch worker for task 475 INFO DataWritingSparkTask: Commit authorized for partition 0 (task 475, attempt 0, stage 139.0) 20/08/03 12:50:50.956 Executor task launch worker for task 475 INFO DataWritingSparkTask: Committed partition 0 (task 475, attempt 0, stage 139.0) 20/08/03 12:50:50.957 Executor task launch worker for task 475 INFO Executor: Finished task 0.0 in stage 139.0 (TID 475). 6472 bytes result sent to driver 20/08/03 12:50:50.957 task-result-getter-3 INFO TaskSetManager: Finished task 0.0 in stage 139.0 (TID 475) in 8 ms on amp-jenkins-worker-06.amp (executor driver) (1/1) 20/08/03 12:50:50.957 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 139.0, whose tasks have all completed, from pool 20/08/03 12:50:50.957 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 139 (testStream$ at LogisticRegressionSuite.scala:39) finished in 0.011 s 20/08/03 12:50:50.958 dag-scheduler-event-loop INFO DAGScheduler: Job 139 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:50.958 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 139: Stage finished 20/08/03 12:50:50.958 stream execution thread for [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f] INFO DAGScheduler: Job 139 finished: testStream$ at LogisticRegressionSuite.scala:39, took 0.013805 s 20/08/03 12:50:50.958 stream execution thread for [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f] INFO WriteToDataSourceV2Exec: Data source write support org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@4abe651a is committing. 20/08/03 12:50:50.958 stream execution thread for [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f] INFO WriteToDataSourceV2Exec: Data source write support org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@4abe651a committed. 20/08/03 12:50:50.964 stream execution thread for [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f] INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-bc0726a9-e9ea-4957-9091-2da4bc53a616/commits/0 using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-bc0726a9-e9ea-4957-9091-2da4bc53a616/commits/.0.370d7d94-8509-478a-b7ea-25d3d2928f25.tmp 20/08/03 12:50:50.972 stream execution thread for [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f] INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-bc0726a9-e9ea-4957-9091-2da4bc53a616/commits/.0.370d7d94-8509-478a-b7ea-25d3d2928f25.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-bc0726a9-e9ea-4957-9091-2da4bc53a616/commits/0 20/08/03 12:50:50.974 stream execution thread for [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f] INFO MicroBatchExecution: Streaming query made progress: { "id" : "48d85991-9680-4d11-a4c1-261bad7ee0ce", "runId" : "2cf71392-a13a-46c8-a8a6-2037b3a96f2f", "name" : null, "timestamp" : "2020-08-03T19:50:50.893Z", "batchId" : 0, "numInputRows" : 100, "inputRowsPerSecond" : 8333.333333333334, "processedRowsPerSecond" : 1250.0, "durationMs" : { "addBatch" : 26, "getBatch" : 0, "latestOffset" : 0, "queryPlanning" : 25, "triggerExecution" : 80, "walCommit" : 13 }, "stateOperators" : [ ], "sources" : [ { "description" : "MemoryStream[_1#1241,_2#1242]", "startOffset" : null, "endOffset" : 0, "numInputRows" : 100, "inputRowsPerSecond" : 8333.333333333334, "processedRowsPerSecond" : 1250.0 } ], "sink" : { "description" : "MemorySink", "numOutputRows" : 100 } } 20/08/03 12:50:50.977 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Asked to cancel job group 2cf71392-a13a-46c8-a8a6-2037b3a96f2f 20/08/03 12:50:50.978 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Asked to cancel job group 2cf71392-a13a-46c8-a8a6-2037b3a96f2f 20/08/03 12:50:50.978 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Query [id = 48d85991-9680-4d11-a4c1-261bad7ee0ce, runId = 2cf71392-a13a-46c8-a8a6-2037b3a96f2f] was stopped 20/08/03 12:50:51.024 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: collect at MLTest.scala:133 20/08/03 12:50:51.159 dag-scheduler-event-loop INFO DAGScheduler: Got job 140 (collect at MLTest.scala:133) with 4 output partitions 20/08/03 12:50:51.160 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 140 (collect at MLTest.scala:133) 20/08/03 12:50:51.160 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:51.160 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:51.160 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 140 (MapPartitionsRDD[492] at collect at MLTest.scala:133), which has no missing parents 20/08/03 12:50:51.162 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_264 stored as values in memory (estimated size 26.0 KiB, free 2.1 GiB) 20/08/03 12:50:51.163 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_264_piece0 stored as bytes in memory (estimated size 10.9 KiB, free 2.1 GiB) 20/08/03 12:50:51.163 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_264_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.9 KiB, free: 2.1 GiB) 20/08/03 12:50:51.164 dag-scheduler-event-loop INFO SparkContext: Created broadcast 264 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:51.164 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 140 (MapPartitionsRDD[492] at collect at MLTest.scala:133) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:51.164 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 140.0 with 4 tasks 20/08/03 12:50:51.165 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 140.0 (TID 476, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:51.165 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 140.0 (TID 477, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:51.165 Executor task launch worker for task 476 INFO Executor: Running task 0.0 in stage 140.0 (TID 476) 20/08/03 12:50:51.165 Executor task launch worker for task 477 INFO Executor: Running task 1.0 in stage 140.0 (TID 477) 20/08/03 12:50:51.170 Executor task launch worker for task 477 INFO Executor: Finished task 1.0 in stage 140.0 (TID 477). 1401 bytes result sent to driver 20/08/03 12:50:51.170 Executor task launch worker for task 476 INFO Executor: Finished task 0.0 in stage 140.0 (TID 476). 1407 bytes result sent to driver 20/08/03 12:50:51.170 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 140.0 (TID 478, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:51.170 Executor task launch worker for task 478 INFO Executor: Running task 2.0 in stage 140.0 (TID 478) 20/08/03 12:50:51.170 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 140.0 (TID 479, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:51.171 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 140.0 (TID 477) in 5 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:51.171 Executor task launch worker for task 479 INFO Executor: Running task 3.0 in stage 140.0 (TID 479) 20/08/03 12:50:51.171 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 140.0 (TID 476) in 6 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:51.173 Executor task launch worker for task 478 INFO Executor: Finished task 2.0 in stage 140.0 (TID 478). 1382 bytes result sent to driver 20/08/03 12:50:51.173 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 140.0 (TID 478) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:51.174 Executor task launch worker for task 479 INFO Executor: Finished task 3.0 in stage 140.0 (TID 479). 1428 bytes result sent to driver 20/08/03 12:50:51.174 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 140.0 (TID 479) in 4 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:51.174 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 140.0, whose tasks have all completed, from pool 20/08/03 12:50:51.174 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 140 (collect at MLTest.scala:133) finished in 0.014 s 20/08/03 12:50:51.174 dag-scheduler-event-loop INFO DAGScheduler: Job 140 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:51.174 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 140: Stage finished 20/08/03 12:50:51.175 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 140 finished: collect at MLTest.scala:133, took 0.016502 s 20/08/03 12:50:51.228 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: collect at MLTest.scala:116 20/08/03 12:50:51.229 dag-scheduler-event-loop INFO DAGScheduler: Got job 141 (collect at MLTest.scala:116) with 4 output partitions 20/08/03 12:50:51.229 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 141 (collect at MLTest.scala:116) 20/08/03 12:50:51.229 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:51.229 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:51.229 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 141 (MapPartitionsRDD[495] at collect at MLTest.scala:116), which has no missing parents 20/08/03 12:50:51.230 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_265 stored as values in memory (estimated size 15.2 KiB, free 2.1 GiB) 20/08/03 12:50:51.232 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_265_piece0 stored as bytes in memory (estimated size 5.8 KiB, free 2.1 GiB) 20/08/03 12:50:51.232 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_265_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 5.8 KiB, free: 2.1 GiB) 20/08/03 12:50:51.232 dag-scheduler-event-loop INFO SparkContext: Created broadcast 265 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:51.232 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 141 (MapPartitionsRDD[495] at collect at MLTest.scala:116) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:51.232 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 141.0 with 4 tasks 20/08/03 12:50:51.233 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 141.0 (TID 480, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:51.233 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 141.0 (TID 481, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:51.233 Executor task launch worker for task 480 INFO Executor: Running task 0.0 in stage 141.0 (TID 480) 20/08/03 12:50:51.233 Executor task launch worker for task 481 INFO Executor: Running task 1.0 in stage 141.0 (TID 481) 20/08/03 12:50:51.239 Executor task launch worker for task 481 INFO Executor: Finished task 1.0 in stage 141.0 (TID 481). 1970 bytes result sent to driver 20/08/03 12:50:51.239 Executor task launch worker for task 480 INFO Executor: Finished task 0.0 in stage 141.0 (TID 480). 1955 bytes result sent to driver 20/08/03 12:50:51.240 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 2.0 in stage 141.0 (TID 482, amp-jenkins-worker-06.amp, executor driver, partition 2, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:51.240 Executor task launch worker for task 482 INFO Executor: Running task 2.0 in stage 141.0 (TID 482) 20/08/03 12:50:51.240 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 3.0 in stage 141.0 (TID 483, amp-jenkins-worker-06.amp, executor driver, partition 3, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:51.240 task-result-getter-1 INFO TaskSetManager: Finished task 1.0 in stage 141.0 (TID 481) in 7 ms on amp-jenkins-worker-06.amp (executor driver) (1/4) 20/08/03 12:50:51.240 task-result-getter-0 INFO TaskSetManager: Finished task 0.0 in stage 141.0 (TID 480) in 7 ms on amp-jenkins-worker-06.amp (executor driver) (2/4) 20/08/03 12:50:51.240 Executor task launch worker for task 483 INFO Executor: Running task 3.0 in stage 141.0 (TID 483) 20/08/03 12:50:51.242 Executor task launch worker for task 482 INFO Executor: Finished task 2.0 in stage 141.0 (TID 482). 1919 bytes result sent to driver 20/08/03 12:50:51.242 Executor task launch worker for task 483 INFO Executor: Finished task 3.0 in stage 141.0 (TID 483). 1856 bytes result sent to driver 20/08/03 12:50:51.242 task-result-getter-2 INFO TaskSetManager: Finished task 2.0 in stage 141.0 (TID 482) in 2 ms on amp-jenkins-worker-06.amp (executor driver) (3/4) 20/08/03 12:50:51.243 task-result-getter-3 INFO TaskSetManager: Finished task 3.0 in stage 141.0 (TID 483) in 3 ms on amp-jenkins-worker-06.amp (executor driver) (4/4) 20/08/03 12:50:51.243 task-result-getter-3 INFO TaskSchedulerImpl: Removed TaskSet 141.0, whose tasks have all completed, from pool 20/08/03 12:50:51.243 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 141 (collect at MLTest.scala:116) finished in 0.014 s 20/08/03 12:50:51.243 dag-scheduler-event-loop INFO DAGScheduler: Job 141 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:51.243 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 141: Stage finished 20/08/03 12:50:51.243 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Job 141 finished: collect at MLTest.scala:116, took 0.015053 s 20/08/03 12:50:51.871 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: StartStream(ProcessingTimeTrigger(0),org.apache.spark.util.SystemClock@46491eae,Map(),null) 20/08/03 12:50:51.872 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Checkpoint root /home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-4be009bb-5eaa-4639-9ce0-5848741dab34 resolved to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-4be009bb-5eaa-4639-9ce0-5848741dab34. 20/08/03 12:50:51.878 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-4be009bb-5eaa-4639-9ce0-5848741dab34/metadata using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-4be009bb-5eaa-4639-9ce0-5848741dab34/.metadata.58da9d6c-45cc-40ef-bb1b-832112e39a45.tmp 20/08/03 12:50:51.885 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-4be009bb-5eaa-4639-9ce0-5848741dab34/.metadata.58da9d6c-45cc-40ef-bb1b-832112e39a45.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-4be009bb-5eaa-4639-9ce0-5848741dab34/metadata 20/08/03 12:50:51.892 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Starting [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745]. Use file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-4be009bb-5eaa-4639-9ce0-5848741dab34 to store the query checkpoint. 20/08/03 12:50:51.894 stream execution thread for [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] INFO MicroBatchExecution: Reading table [org.apache.spark.sql.execution.streaming.MemoryStreamTable@26107a4b] from DataSourceV2 named 'memory' [org.apache.spark.sql.execution.streaming.MemoryStreamTableProvider$@490dc5fd] 20/08/03 12:50:51.895 stream execution thread for [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] INFO MicroBatchExecution: Starting new streaming query. 20/08/03 12:50:51.896 stream execution thread for [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] INFO MicroBatchExecution: Stream started from {} 20/08/03 12:50:51.896 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: AddData to MemoryStream[_1#1321,_2#1322]: (0.0,[6.7885086340489185,3.4576551565453197]),(0.0,[5.056501247252454,2.574601118217243]),(0.0,[6.075652199068211,3.355341616104334]),(0.0,[5.166332778860912,2.448377481827237]),(0.0,[5.684896098313557,3.7046548957911765]),(0.0,[6.50731807165581,3.004047684427792]),(0.0,[7.010912781425465,2.7780024496013715]),(1.0,[4.8414027004203835,3.211158821420722]),(1.0,[5.436985752813093,3.296990772729685]),(2.0,[4.84644814192866,3.1968908375605634]),(0.0,[7.127365231828297,3.248100610649046]),(1.0,[6.241767847456931,3.7192174938869087]),(0.0,[5.613780653545803,3.020421477260098]),(0.0,[6.882841862374561,2.915254192504282]),(1.0,[5.699513929932456,2.247074740074896]),(0.0,[7.02192590749258,2.4627220581296783]),(1.0,[4.216237488061392,2.656699174446591]),(0.0,[3.7324966114897262,2.3462678397985552]),(1.0,[5.741978867831515,3.6187835154633294]),(0.0,[5.620150427156592,3.1692083626745053]),(0.0,[5.578107805726126,2.2863316707315384]),(0.0,[5.442698469962282,2.834758817537672]),(0.0,[6.767616345179818,3.0391562262408645]),(1.0,[4.925837281912354,3.8685534349585087]),(0.0,[6.791684731921275,2.595654759591007]),(0.0,[6.731122446118569,2.2151938610676063]),(0.0,[6.092251787382735,3.1648644467226514]),(0.0,[7.007311457155493,2.3945255550358033]),(1.0,[6.067332171369262,3.301578503282249]),(0.0,[5.404295990288988,3.291892487883425]),(0.0,[7.674839072483042,2.781183265589518]),(0.0,[4.2837121498611985,3.2254558585309234]),(1.0,[6.112789784401108,2.664233955695323]),(1.0,[5.840780678327939,2.8504603244268845]),(1.0,[5.389268793409797,3.0486766295260552]),(0.0,[7.206579880655595,2.5745696091376855]),(1.0,[6.334680742393493,2.989165223067271]),(0.0,[6.39220828708044,3.5916060870142212]),(1.0,[6.574935571365726,3.1536220523404945]),(0.0,[6.604522213141694,2.734798375932801]),(0.0,[6.508320460023237,2.946338492652808]),(1.0,[5.210534043458435,2.24447246587153]),(1.0,[4.301820482847355,2.8057320879830776]),(0.0,[5.504349176848336,3.165273107673573]),(0.0,[5.444799426865632,3.251448135769693]),(0.0,[5.473220085163719,3.076888207026409]),(0.0,[6.426387726380042,2.938545841808128]),(0.0,[5.909860769315067,3.6056829798513315]),(0.0,[6.475193522926937,3.807440177850298]),(0.0,[5.963848054893707,2.623550555268978]),(1.0,[6.009222629896281,3.4475575899620026]),(0.0,[7.094718114371827,2.4454009956067817]),(1.0,[4.264010706382008,3.65338684298196]),(0.0,[5.379585321764099,3.381481745785779]),(0.0,[7.146010317633724,3.5485733241937085]),(0.0,[5.683954590364761,2.0745476381163566]),(0.0,[6.2072015922395885,3.3432172748930773]),(0.0,[5.226174068792373,3.579701566179305]),(0.0,[7.316647506340967,3.0863730559231892]),(1.0,[6.915325522847731,2.803084035949653]),(0.0,[6.479630329902845,3.239007840935509]),(0.0,[5.454587096334029,3.2374754911843655]),(2.0,[6.391989224379249,3.434901891800874]),(1.0,[6.0886711934815265,3.1971117650588363]),(0.0,[7.684738047156347,3.421241896031828]),(0.0,[4.85493416785451,3.1087723328558345]),(0.0,[6.305063368354057,2.2060572623972567]),(0.0,[7.081748274709549,2.724685616028466]),(0.0,[6.584215742308165,2.9773965559705675]),(1.0,[6.612853131358183,2.8663569212164877]),(0.0,[5.756132728856622,2.7905503944752907]),(1.0,[6.135092550527577,2.5582480777704872]),(1.0,[5.301620953306969,4.223612451290534]),(1.0,[5.9682958432514,3.8618929524452805]),(2.0,[6.207939723169083,3.439309516984485]),(0.0,[6.627109164472448,2.7520560194737964]),(0.0,[3.9259860466490686,3.0983115739148284]),(0.0,[6.0020085765576665,2.6176816137758694]),(0.0,[4.062185373981144,3.76166658403873]),(1.0,[6.2962118879304585,3.764267301123263]),(1.0,[6.14908975112769,3.590411745294196]),(1.0,[6.567608573964195,2.9306232708439306]),(0.0,[6.201805185587324,3.4958144403354416]),(1.0,[7.319347971525034,3.648290394729337]),(1.0,[6.212234548496149,3.6633867263201436]),(0.0,[5.870287402238878,2.7075846255583254]),(0.0,[5.8287726136817,2.483308333263797]),(0.0,[5.819965967504458,3.9163413775672242]),(0.0,[6.982232083397012,2.503264912876912]),(0.0,[6.084806791595145,3.72518246843444]),(0.0,[5.92962672345617,2.484757315931346]),(1.0,[7.3168542953228615,3.229242214881765]),(0.0,[5.428611939474008,2.1676964414169606]),(1.0,[6.74298764459546,2.6034237914698872]),(0.0,[7.467271990105523,3.4604682039447336]),(0.0,[6.970465097926772,4.3162388355161205]),(0.0,[5.436435936556991,2.747513090695181]),(0.0,[7.143361618211275,2.6898992927088976]),(0.0,[5.568158116555633,2.316684365492898]),(0.0,[5.40320969646403,1.929404297480336]) 20/08/03 12:50:51.897 stream execution thread for [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] INFO MicroBatchExecution: Streaming query made progress: { "id" : "19e84854-89a6-45dc-b264-48c149f28ce8", "runId" : "6391c38a-cd1e-4efe-953e-784ff4ee3745", "name" : null, "timestamp" : "2020-08-03T19:50:51.895Z", "batchId" : 0, "numInputRows" : 0, "processedRowsPerSecond" : 0.0, "durationMs" : { "latestOffset" : 0, "triggerExecution" : 1 }, "stateOperators" : [ ], "sources" : [ { "description" : "MemoryStream[_1#1321,_2#1322]", "startOffset" : null, "endOffset" : null, "numInputRows" : 0, "processedRowsPerSecond" : 0.0 } ], "sink" : { "description" : "MemorySink", "numOutputRows" : -1 } } 20/08/03 12:50:51.899 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO LogisticRegressionSuite: Processing test stream action: CheckAnswerByFunc 20/08/03 12:50:51.911 stream execution thread for [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-4be009bb-5eaa-4639-9ce0-5848741dab34/offsets/0 using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-4be009bb-5eaa-4639-9ce0-5848741dab34/offsets/.0.0dcccacb-f681-4be7-a143-2e1560376b87.tmp 20/08/03 12:50:51.918 stream execution thread for [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-4be009bb-5eaa-4639-9ce0-5848741dab34/offsets/.0.0dcccacb-f681-4be7-a143-2e1560376b87.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-4be009bb-5eaa-4639-9ce0-5848741dab34/offsets/0 20/08/03 12:50:51.918 stream execution thread for [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] INFO MicroBatchExecution: Committed offsets for batch 0. Metadata OffsetSeqMetadata(0,1596484251907,Map(spark.sql.streaming.stateStore.providerClass -> org.apache.spark.sql.execution.streaming.state.HDFSBackedStateStoreProvider, spark.sql.streaming.join.stateFormatVersion -> 2, spark.sql.streaming.flatMapGroupsWithState.stateFormatVersion -> 2, spark.sql.streaming.multipleWatermarkPolicy -> min, spark.sql.streaming.aggregation.stateFormatVersion -> 2, spark.sql.shuffle.partitions -> 5)) 20/08/03 12:50:51.968 stream execution thread for [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] INFO CodeGenerator: Code generated in 17.122353 ms 20/08/03 12:50:51.972 stream execution thread for [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] INFO WriteToDataSourceV2Exec: Start processing data source write support: org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@69ea8ac2. The input RDD has 1 partitions. 20/08/03 12:50:51.972 stream execution thread for [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] INFO SparkContext: Starting job: testStream$ at LogisticRegressionSuite.scala:39 20/08/03 12:50:51.973 dag-scheduler-event-loop INFO DAGScheduler: Got job 142 (testStream$ at LogisticRegressionSuite.scala:39) with 1 output partitions 20/08/03 12:50:51.974 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 142 (testStream$ at LogisticRegressionSuite.scala:39) 20/08/03 12:50:51.974 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:51.974 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:51.974 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 142 (MapPartitionsRDD[498] at testStream$ at LogisticRegressionSuite.scala:39), which has no missing parents 20/08/03 12:50:51.976 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_266 stored as values in memory (estimated size 21.1 KiB, free 2.1 GiB) 20/08/03 12:50:51.977 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_266_piece0 stored as bytes in memory (estimated size 9.6 KiB, free 2.1 GiB) 20/08/03 12:50:51.977 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_266_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 9.6 KiB, free: 2.1 GiB) 20/08/03 12:50:51.977 dag-scheduler-event-loop INFO SparkContext: Created broadcast 266 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:51.977 dag-scheduler-event-loop INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 142 (MapPartitionsRDD[498] at testStream$ at LogisticRegressionSuite.scala:39) (first 15 tasks are for partitions Vector(0)) 20/08/03 12:50:51.977 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 142.0 with 1 tasks 20/08/03 12:50:51.978 dispatcher-event-loop-1 INFO TaskSetManager: Starting task 0.0 in stage 142.0 (TID 484, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 18876 bytes) 20/08/03 12:50:51.978 Executor task launch worker for task 484 INFO Executor: Running task 0.0 in stage 142.0 (TID 484) 20/08/03 12:50:51.988 Executor task launch worker for task 484 INFO DataWritingSparkTask: Commit authorized for partition 0 (task 484, attempt 0, stage 142.0) 20/08/03 12:50:51.989 Executor task launch worker for task 484 INFO DataWritingSparkTask: Committed partition 0 (task 484, attempt 0, stage 142.0) 20/08/03 12:50:51.989 Executor task launch worker for task 484 INFO Executor: Finished task 0.0 in stage 142.0 (TID 484). 6472 bytes result sent to driver 20/08/03 12:50:51.990 task-result-getter-1 INFO TaskSetManager: Finished task 0.0 in stage 142.0 (TID 484) in 12 ms on amp-jenkins-worker-06.amp (executor driver) (1/1) 20/08/03 12:50:51.990 task-result-getter-1 INFO TaskSchedulerImpl: Removed TaskSet 142.0, whose tasks have all completed, from pool 20/08/03 12:50:51.990 dag-scheduler-event-loop INFO DAGScheduler: ResultStage 142 (testStream$ at LogisticRegressionSuite.scala:39) finished in 0.015 s 20/08/03 12:50:51.990 dag-scheduler-event-loop INFO DAGScheduler: Job 142 is finished. Cancelling potential speculative or zombie tasks for this job 20/08/03 12:50:51.990 dag-scheduler-event-loop INFO TaskSchedulerImpl: Killing all running tasks in stage 142: Stage finished 20/08/03 12:50:51.990 stream execution thread for [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] INFO DAGScheduler: Job 142 finished: testStream$ at LogisticRegressionSuite.scala:39, took 0.017994 s 20/08/03 12:50:51.991 stream execution thread for [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] INFO WriteToDataSourceV2Exec: Data source write support org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@69ea8ac2 is committing. 20/08/03 12:50:51.991 stream execution thread for [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] INFO WriteToDataSourceV2Exec: Data source write support org.apache.spark.sql.execution.streaming.sources.MicroBatchWrite@69ea8ac2 committed. 20/08/03 12:50:51.996 stream execution thread for [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] INFO CheckpointFileManager: Writing atomically to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-4be009bb-5eaa-4639-9ce0-5848741dab34/commits/0 using temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-4be009bb-5eaa-4639-9ce0-5848741dab34/commits/.0.47a0c5d2-2fd9-4ca7-b278-41f5508a1ad6.tmp 20/08/03 12:50:52.003 stream execution thread for [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] INFO CheckpointFileManager: Renamed temp file file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-4be009bb-5eaa-4639-9ce0-5848741dab34/commits/.0.47a0c5d2-2fd9-4ca7-b278-41f5508a1ad6.tmp to file:/home/jenkins/workspace/NewSparkPullRequestBuilder/mllib/target/tmp/org.apache.spark.ml.classification.LogisticRegressionSuite/streaming.metadata-4be009bb-5eaa-4639-9ce0-5848741dab34/commits/0 20/08/03 12:50:52.004 stream execution thread for [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] INFO MicroBatchExecution: Streaming query made progress: { "id" : "19e84854-89a6-45dc-b264-48c149f28ce8", "runId" : "6391c38a-cd1e-4efe-953e-784ff4ee3745", "name" : null, "timestamp" : "2020-08-03T19:50:51.907Z", "batchId" : 0, "numInputRows" : 100, "inputRowsPerSecond" : 8333.333333333334, "processedRowsPerSecond" : 1041.6666666666667, "durationMs" : { "addBatch" : 49, "getBatch" : 0, "latestOffset" : 0, "queryPlanning" : 23, "triggerExecution" : 96, "walCommit" : 11 }, "stateOperators" : [ ], "sources" : [ { "description" : "MemoryStream[_1#1321,_2#1322]", "startOffset" : null, "endOffset" : 0, "numInputRows" : 100, "inputRowsPerSecond" : 8333.333333333334, "processedRowsPerSecond" : 1041.6666666666667 } ], "sink" : { "description" : "MemorySink", "numOutputRows" : 100 } } 20/08/03 12:50:52.005 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Asked to cancel job group 6391c38a-cd1e-4efe-953e-784ff4ee3745 20/08/03 12:50:52.006 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO DAGScheduler: Asked to cancel job group 6391c38a-cd1e-4efe-953e-784ff4ee3745 20/08/03 12:50:52.006 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO MicroBatchExecution: Query [id = 19e84854-89a6-45dc-b264-48c149f28ce8, runId = 6391c38a-cd1e-4efe-953e-784ff4ee3745] was stopped 20/08/03 12:50:52.061 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO CodeGenerator: Code generated in 19.520974 ms 20/08/03 12:50:52.069 pool-1-thread-1-ScalaTest-running-LogisticRegressionSuite INFO SparkContext: Starting job: collect at MLTest.scala:133 20/08/03 12:50:52.070 dag-scheduler-event-loop INFO DAGScheduler: Got job 143 (collect at MLTest.scala:133) with 4 output partitions 20/08/03 12:50:52.070 dag-scheduler-event-loop INFO DAGScheduler: Final stage: ResultStage 143 (collect at MLTest.scala:133) 20/08/03 12:50:52.070 dag-scheduler-event-loop INFO DAGScheduler: Parents of final stage: List() 20/08/03 12:50:52.070 dag-scheduler-event-loop INFO DAGScheduler: Missing parents: List() 20/08/03 12:50:52.070 dag-scheduler-event-loop INFO DAGScheduler: Submitting ResultStage 143 (MapPartitionsRDD[501] at collect at MLTest.scala:133), which has no missing parents 20/08/03 12:50:52.072 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_267 stored as values in memory (estimated size 24.6 KiB, free 2.1 GiB) 20/08/03 12:50:52.073 dag-scheduler-event-loop INFO MemoryStore: Block broadcast_267_piece0 stored as bytes in memory (estimated size 10.6 KiB, free 2.1 GiB) 20/08/03 12:50:52.073 dispatcher-BlockManagerMaster INFO BlockManagerInfo: Added broadcast_267_piece0 in memory on amp-jenkins-worker-06.amp:34426 (size: 10.6 KiB, free: 2.1 GiB) 20/08/03 12:50:52.073 dag-scheduler-event-loop INFO SparkContext: Created broadcast 267 from broadcast at DAGScheduler.scala:1221 20/08/03 12:50:52.073 dag-scheduler-event-loop INFO DAGScheduler: Submitting 4 missing tasks from ResultStage 143 (MapPartitionsRDD[501] at collect at MLTest.scala:133) (first 15 tasks are for partitions Vector(0, 1, 2, 3)) 20/08/03 12:50:52.073 dag-scheduler-event-loop INFO TaskSchedulerImpl: Adding task set 143.0 with 4 tasks 20/08/03 12:50:52.074 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 0.0 in stage 143.0 (TID 485, amp-jenkins-worker-06.amp, executor driver, partition 0, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:52.074 dispatcher-event-loop-0 INFO TaskSetManager: Starting task 1.0 in stage 143.0 (TID 486, amp-jenkins-worker-06.amp, executor driver, partition 1, PROCESS_LOCAL, 8707 bytes) 20/08/03 12:50:52.074 Executor task launch worker for task 485 INFO Executor: Running task 0.0 in stage 143.0 (TID 485) 20/08/03 12:50:52.074 Executor task launch worker for task 486 INFO Executor: Running task 1.0 in stage 143.0 (TID 486) 20